Jul 05 11:53:34.239298 ip-10-0-143-202 systemd[1]: Starting Kubernetes Kubelet... Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.649764 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657347 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657361 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657364 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657367 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657371 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657374 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657376 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:34.701212 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657379 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657382 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657384 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657387 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657389 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657392 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657395 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657397 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657400 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657402 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657405 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657407 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657410 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657412 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657415 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657419 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657423 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657426 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657429 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:34.702191 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657432 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657434 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657438 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657441 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657444 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657453 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657456 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657458 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657461 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657463 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657466 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657468 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657471 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657473 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657476 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657478 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657481 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657484 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657487 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657489 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:34.702783 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657492 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657494 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657497 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657499 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657502 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657504 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657507 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657509 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657513 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657517 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657520 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657523 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657525 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657527 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657531 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657535 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657537 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657540 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657542 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657545 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:34.703508 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657547 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657550 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657553 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657556 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657558 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657561 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657563 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657566 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657569 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657572 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657574 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657577 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657579 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657582 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657585 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657587 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657590 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657593 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657595 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657598 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:34.704116 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657601 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657603 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657606 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657608 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657611 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657614 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657618 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657621 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657624 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657627 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657630 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657633 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.657635 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658335 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658343 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658346 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658348 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658352 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658354 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658357 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:34.704816 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658360 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658362 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658365 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658367 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658370 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658373 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658376 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658378 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658382 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658386 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658389 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658392 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658395 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658397 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658400 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658402 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658405 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658407 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658410 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658413 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:34.705380 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658416 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658418 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658421 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658423 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658426 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658428 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658431 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658434 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658438 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658441 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658444 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658447 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658449 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658452 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658455 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658457 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658460 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658462 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658465 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658467 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:34.705977 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658469 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658472 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658474 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658478 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658480 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658483 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658487 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658490 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658493 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658496 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658498 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658502 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658504 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658509 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658511 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658514 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658517 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658519 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658522 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:34.706549 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658525 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658528 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658531 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658534 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658537 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658540 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658542 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658545 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658548 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658551 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658553 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658556 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658559 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658561 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658564 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658567 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658570 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658573 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658576 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658578 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:34.707160 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658581 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658584 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658586 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658589 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658591 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658594 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658596 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658599 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658601 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658604 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658606 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658609 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658611 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658976 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658981 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658984 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658987 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658990 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:34.707690 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658992 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658995 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.658997 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659000 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659002 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659005 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659008 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659010 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659013 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659016 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659019 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659043 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659046 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659049 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659051 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659054 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659057 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659059 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659063 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659065 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:34.708217 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659068 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659070 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659072 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659075 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659078 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659081 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659084 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659086 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659089 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659091 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659094 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659098 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659102 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659105 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659107 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659110 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659113 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659116 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659118 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659121 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:34.708789 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659124 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659127 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659129 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659133 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659136 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659139 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659142 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659145 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659150 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659153 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659155 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659158 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659161 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659164 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659166 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659169 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659172 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659174 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659177 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659179 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:34.709399 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659182 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659186 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659189 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659191 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659194 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659196 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659199 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659201 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659204 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659206 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659209 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659211 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659214 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659218 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659220 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659223 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659225 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659229 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659231 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659234 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:34.709948 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659236 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659239 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659241 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659244 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659248 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659250 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659253 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659255 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659258 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659261 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659265 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659267 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659270 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659274 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659345 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659352 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659359 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659363 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659367 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659370 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659374 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 05 11:53:34.906163 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659378 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 05 11:53:34.724367 ip-10-0-143-202 systemd[1]: Started Kubernetes Kubelet. Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659381 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659384 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659387 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659391 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659394 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659397 2574 flags.go:64] FLAG: --cgroup-root="" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659400 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659403 2574 flags.go:64] FLAG: --client-ca-file="" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659406 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659409 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659413 2574 flags.go:64] FLAG: --cluster-domain="" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659416 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659419 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659422 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659425 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659429 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659431 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659434 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659438 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659441 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659443 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659446 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659449 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 05 11:53:34.919470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659453 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659457 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659460 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659463 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659466 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659469 2574 flags.go:64] FLAG: --enable-server="true" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659471 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659476 2574 flags.go:64] FLAG: --event-burst="100" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659479 2574 flags.go:64] FLAG: --event-qps="50" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659482 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659484 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659487 2574 flags.go:64] FLAG: --eviction-hard="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659492 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659495 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659498 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659501 2574 flags.go:64] FLAG: --eviction-soft="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659503 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659506 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659509 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659512 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659515 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659518 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659521 2574 flags.go:64] FLAG: --feature-gates="" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659524 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659527 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 05 11:53:34.920512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659530 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659533 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659536 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659539 2574 flags.go:64] FLAG: --help="false" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659541 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-202.ec2.internal" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659544 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659547 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659551 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659554 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659558 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659560 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659563 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659566 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659569 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659571 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659574 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659577 2574 flags.go:64] FLAG: --kube-reserved="" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659580 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659583 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659587 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659590 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659593 2574 flags.go:64] FLAG: --lock-file="" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659595 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659598 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 05 11:53:34.923621 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659601 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659606 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659609 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659612 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659615 2574 flags.go:64] FLAG: --logging-format="text" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659618 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659621 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659624 2574 flags.go:64] FLAG: --manifest-url="" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659626 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659630 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659633 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659637 2574 flags.go:64] FLAG: --max-pods="110" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659640 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659643 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659645 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659650 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659653 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659655 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659658 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659665 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659668 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659671 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659674 2574 flags.go:64] FLAG: --pod-cidr="" Jul 05 11:53:34.924419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659677 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659683 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659686 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659689 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659691 2574 flags.go:64] FLAG: --port="10250" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659696 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659699 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0750ca29707c70c2e" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659702 2574 flags.go:64] FLAG: --qos-reserved="" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659705 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659707 2574 flags.go:64] FLAG: --register-node="true" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659710 2574 flags.go:64] FLAG: --register-with-taints="" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659714 2574 flags.go:64] FLAG: --registry-burst="10" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659717 2574 flags.go:64] FLAG: --registry-qps="5" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659720 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659722 2574 flags.go:64] FLAG: --reserved-memory="" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659726 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659729 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659732 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659735 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659737 2574 flags.go:64] FLAG: --runonce="false" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659740 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659743 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659746 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659749 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659753 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659756 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 05 11:53:34.925401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659759 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659762 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659764 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659767 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659770 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659773 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659776 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659778 2574 flags.go:64] FLAG: --system-cgroups="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659781 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659786 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659789 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659795 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659799 2574 flags.go:64] FLAG: --tls-min-version="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659802 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659804 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659807 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659810 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659813 2574 flags.go:64] FLAG: --v="2" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659817 2574 flags.go:64] FLAG: --version="false" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659821 2574 flags.go:64] FLAG: --vmodule="" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659824 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.659828 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659956 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659960 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:34.926214 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659963 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659965 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659968 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659971 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659974 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659977 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659981 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659983 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659986 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659990 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659993 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659996 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.659999 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660002 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660005 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660008 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660010 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660013 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660016 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660020 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:34.927001 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660038 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660041 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660044 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660046 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660049 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660053 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660056 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660059 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660062 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660065 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660068 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660070 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660073 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660076 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660078 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660081 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660083 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660085 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660089 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:34.927605 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660092 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660095 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660097 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660100 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660102 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660105 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660107 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660110 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660112 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660115 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660118 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660120 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660124 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660127 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660130 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660133 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660135 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660138 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660140 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660143 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:34.928383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660146 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660148 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660151 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660154 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660156 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660159 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660161 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660164 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660166 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660169 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660172 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660178 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660181 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660183 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660186 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660188 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660191 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660193 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660196 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660199 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:34.928962 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660201 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660204 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660206 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660209 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660213 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660216 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660218 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660222 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660225 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660228 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660231 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660233 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660236 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660239 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660242 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660245 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660248 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:36.569699 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.660250 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.661098 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.670369 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.670383 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.671133 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.671140 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671199 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671205 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671208 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671211 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671214 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671217 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671219 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671222 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671225 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:36.570550 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671231 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671235 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671237 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671240 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671243 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671246 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671249 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671251 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671254 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671256 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671259 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671262 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671265 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671268 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671270 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671274 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671276 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671279 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671282 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671284 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:36.571197 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671287 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671289 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671292 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671294 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671297 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671299 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671302 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671304 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671307 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671309 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671312 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671314 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671318 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671325 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671327 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671330 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671332 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671335 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671337 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671340 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:36.571826 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671342 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671345 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671347 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671350 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671353 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671355 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671359 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671362 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671366 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671369 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671372 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671374 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671377 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671381 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671383 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671386 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671388 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671391 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671393 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:36.572412 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671396 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671398 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671401 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671403 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671407 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671411 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671414 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671417 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671420 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671423 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671425 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671428 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671431 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671433 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671436 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671438 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671441 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671444 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671448 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671451 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:36.573383 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671454 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671457 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671460 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671462 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671465 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671468 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671470 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671472 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671475 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671477 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671480 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.671485 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671590 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671595 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671598 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 05 11:53:36.573968 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671601 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671603 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671608 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671611 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671615 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671618 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671620 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671623 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671625 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671628 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671631 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671634 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671637 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671639 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671641 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671644 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671647 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671649 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671652 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 05 11:53:36.574472 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671656 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671662 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671666 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671670 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671673 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671675 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671678 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671681 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671685 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671688 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671690 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671693 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671695 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671698 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671701 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671707 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671710 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671713 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671716 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 05 11:53:36.575106 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671719 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671721 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671724 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671726 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671729 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671731 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671734 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671736 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671739 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671741 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671744 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671747 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671749 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671751 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671754 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671757 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671760 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671762 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671765 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671767 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 05 11:53:36.575648 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671770 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671772 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671775 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671777 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671780 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671782 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671785 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671787 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671791 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671793 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671797 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671800 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671802 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671805 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671808 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671810 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671813 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671815 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671817 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671820 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 05 11:53:36.576225 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671823 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671825 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671827 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671830 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671833 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671835 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671838 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671841 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671844 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671847 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671849 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671852 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671854 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671857 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671859 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671862 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671864 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 05 11:53:36.576922 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:34.671867 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.671872 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.671970 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.673949 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.675254 2574 server.go:1019] "Starting client certificate rotation" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.675352 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.675381 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.677543 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.678664 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.691040 2574 log.go:25] "Validated CRI v1 runtime API" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.694202 2574 log.go:25] "Validated CRI v1 image API" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.695798 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 05 11:53:36.577399 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.701147 2574 fs.go:135] Filesystem UUIDs: map[768332a0-0cd7-4efb-ac47-b222a86f23b7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 83f656e5-1953-4719-be35-5a24dd4b31a4:/dev/nvme0n1p3] Jul 05 11:53:36.577862 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.701161 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.706107 2574 manager.go:216] Machine: {Timestamp:2026-07-05 11:53:34.704781383 +0000 UTC m=+0.351726384 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3074135 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23a71961d4acdb8a0bfb7a7497fa77 SystemUUID:ec23a719-61d4-acdb-8a0b-fb7a7497fa77 BootID:08b9261f-e487-411a-96c7-03737a7bdf6e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:96:56:60:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:96:56:60:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:44:80:64:3f:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.706209 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.706316 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.706636 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.707194 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.707218 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 05 11:53:36.577967 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.708292 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.708303 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.708311 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.708759 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.709913 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712358 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712460 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712485 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712495 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712513 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712521 2574 kubelet.go:398] "Adding apiserver pod source" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712530 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712862 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.712876 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.717232 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.718985 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.719014 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720070 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720084 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720092 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720097 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720111 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720121 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720130 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720136 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720142 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720147 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720156 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720163 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720191 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.720196 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.723581 2574 server.go:1268] "Started kubelet" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.723692 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.724214 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.724298 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.724439 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.724959 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.724973 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.728801 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 05 11:53:36.795452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.729177 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.729471 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.734361 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.734379 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.734928 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.735157 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.735171 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.735730 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.735739 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.735783 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf2vw" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.734678 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.736589 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zf2vw" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.736784 2574 factory.go:55] Registering systemd factory Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.736797 2574 factory.go:223] Registration of the systemd container factory successfully Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.736993 2574 factory.go:153] Registering CRI-O factory Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.737006 2574 factory.go:223] Registration of the crio container factory successfully Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.737070 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.737098 2574 factory.go:103] Registering Raw factory Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.737114 2574 manager.go:1196] Started watching for new ooms in manager Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.737514 2574 manager.go:349] Starting recovery of all containers Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.742164 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.745369 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-202.ec2.internal\" not found" node="ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.746729 2574 manager.go:354] Recovery completed Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.748652 2574 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.753052 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.754821 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.754846 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.754857 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.755385 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.755395 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.755414 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.755430 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756131 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756158 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756172 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756396 2574 policy_none.go:49] "None policy: Start" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756829 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.756843 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.757730 2574 policy_none.go:47] "Start" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.783643 2574 manager.go:337] "Starting Device Plugin manager" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.783675 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.783687 2574 server.go:87] "Starting device plugin registration server" Jul 05 11:53:36.797339 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.783925 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.783942 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.784140 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.784238 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.784248 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.784562 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.784601 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.847685 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.849041 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.849098 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.849191 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.849254 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.851520 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.884169 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.884995 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.885043 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.885060 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.885083 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.894144 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.894162 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-202.ec2.internal\": node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:34.906018 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.950205 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal"] Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.950285 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.951076 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.951099 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.951109 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.952362 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.952472 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.952495 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.952978 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.952984 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.953005 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.953008 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.953017 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.953019 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.798524 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.954144 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.954165 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.954707 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientMemory" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.954732 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasNoDiskPressure" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:34.954742 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeHasSufficientPID" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.381707 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-202.ec2.internal\" not found" node="ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.385788 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-202.ec2.internal\" not found" node="ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.395616 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.426630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.426651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.426668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71665d816167a69ba4f61ac61f31e766-config\") pod \"kube-apiserver-proxy-ip-10-0-143-202.ec2.internal\" (UID: \"71665d816167a69ba4f61ac61f31e766\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.495698 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527109 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71665d816167a69ba4f61ac61f31e766-config\") pod \"kube-apiserver-proxy-ip-10-0-143-202.ec2.internal\" (UID: \"71665d816167a69ba4f61ac61f31e766\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527172 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/71665d816167a69ba4f61ac61f31e766-config\") pod \"kube-apiserver-proxy-ip-10-0-143-202.ec2.internal\" (UID: \"71665d816167a69ba4f61ac61f31e766\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527186 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.527192 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71246e965b265fa9911eff2ab0b350b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal\" (UID: \"71246e965b265fa9911eff2ab0b350b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799476 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.596346 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.799992 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.684888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799992 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:36.688492 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:36.799992 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.697533 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:36.799992 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.797960 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.001250 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.898450 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.001250 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:36.999641 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.063808 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.063933 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.063970 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:37.100419 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.127148 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-04 11:48:36 +0000 UTC" deadline="2028-03-05 03:19:02.841562176 +0000 UTC" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.127193 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14607h25m25.714373309s" Jul 05 11:53:37.177663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.141938 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:37.201120 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:37.227278 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71246e965b265fa9911eff2ab0b350b8.slice/crio-ab9d315b9d5f28d08cb6b3b6cf7a36ee76ee11f9481063e7e18ae2f15f240b94 WatchSource:0}: Error finding container ab9d315b9d5f28d08cb6b3b6cf7a36ee76ee11f9481063e7e18ae2f15f240b94: Status 404 returned error can't find the container with id ab9d315b9d5f28d08cb6b3b6cf7a36ee76ee11f9481063e7e18ae2f15f240b94 Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:37.227853 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71665d816167a69ba4f61ac61f31e766.slice/crio-612814a0b1a3b7c2879f56d3ee088c392f521dabf2687eb0cb7d244423d3db70 WatchSource:0}: Error finding container 612814a0b1a3b7c2879f56d3ee088c392f521dabf2687eb0cb7d244423d3db70: Status 404 returned error can't find the container with id 612814a0b1a3b7c2879f56d3ee088c392f521dabf2687eb0cb7d244423d3db70 Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.233132 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.241638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" event={"ID":"71665d816167a69ba4f61ac61f31e766","Type":"ContainerStarted","Data":"612814a0b1a3b7c2879f56d3ee088c392f521dabf2687eb0cb7d244423d3db70"} Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.242507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" event={"ID":"71246e965b265fa9911eff2ab0b350b8","Type":"ContainerStarted","Data":"ab9d315b9d5f28d08cb6b3b6cf7a36ee76ee11f9481063e7e18ae2f15f240b94"} Jul 05 11:53:37.362833 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:37.301833 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.597694 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:37.402348 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.597694 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:37.502798 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-202.ec2.internal\" not found" Jul 05 11:53:37.597694 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.555748 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:37.703808 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.625074 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" Jul 05 11:53:37.703808 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.634700 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 11:53:37.703808 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.635788 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" Jul 05 11:53:37.703808 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:37.643493 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 05 11:53:38.102018 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.101937 2574 apiserver.go:52] "Watching apiserver" Jul 05 11:53:38.112333 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.112309 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 05 11:53:38.113536 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.113508 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-wjljl","openshift-multus/multus-vjs65","openshift-multus/network-metrics-daemon-29x85","openshift-network-diagnostics/network-check-target-tsfb7","openshift-network-operator/iptables-alerter-kstgl","kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm","openshift-cluster-node-tuning-operator/tuned-jkkph","openshift-dns/node-resolver-f6l9g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-djdvv","kube-system/global-pull-secret-syncer-prgqv","kube-system/konnectivity-agent-jvkdp","openshift-image-registry/node-ca-x7tsl"] Jul 05 11:53:38.116119 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.115861 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.117998 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.117966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.118464 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.118434 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.119008 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.119240 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.119329 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t6sz5\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.120296 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.120539 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6c8mp\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.120574 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 05 11:53:38.121226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.121133 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.121636 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.121275 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:38.123984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.123964 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 05 11:53:38.124069 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.124035 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:38.124125 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.124100 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:38.125991 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.125961 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.128275 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.128205 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.128370 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.128300 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 05 11:53:38.128996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.128975 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.129105 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.128975 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t6ct2\"" Jul 05 11:53:38.129307 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.129290 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.130434 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.130418 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.130901 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.130879 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 05 11:53:38.130991 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.130934 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.131326 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.131308 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.131465 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.131309 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jfpp7\"" Jul 05 11:53:38.132976 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.132864 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.132976 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.132911 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nl7dm\"" Jul 05 11:53:38.133175 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.133154 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.133585 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.133566 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.134645 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134557 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-registration-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.134645 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2xg\" (UniqueName: \"kubernetes.io/projected/6f110573-9832-4310-93b6-1bc140a48214-kube-api-access-qd2xg\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.134645 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134619 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e98aea-c00e-4411-a3f1-f4d332dd5959-konnectivity-ca\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134741 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134814 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43583d36-b10f-4976-8829-03f8a1b1b7ea-iptables-alerter-script\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-socket-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.134863 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134857 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e98aea-c00e-4411-a3f1-f4d332dd5959-agent-certs\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.135146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134892 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-device-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.135146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-sys-fs\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.135146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltk4m\" (UniqueName: \"kubernetes.io/projected/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kube-api-access-ltk4m\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.135146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134959 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43583d36-b10f-4976-8829-03f8a1b1b7ea-host-slash\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.135146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.134999 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j267d\" (UniqueName: \"kubernetes.io/projected/43583d36-b10f-4976-8829-03f8a1b1b7ea-kube-api-access-j267d\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.135865 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.135843 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.136546 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.136524 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 05 11:53:38.136742 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.136727 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 05 11:53:38.136847 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.136831 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5qcq\"" Jul 05 11:53:38.137227 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.137097 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.137227 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.137116 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 05 11:53:38.137227 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.137136 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.137417 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.137329 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 05 11:53:38.138405 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.138387 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 05 11:53:38.138722 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.138525 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rh2dl\"" Jul 05 11:53:38.138858 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.138795 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.141136 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.140967 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.141682 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.141262 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.141682 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.141383 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.141682 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.141418 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jql6j\"" Jul 05 11:53:38.143714 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.143696 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.143807 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.143782 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:38.144689 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.144668 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 05 11:53:38.145039 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145022 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 05 11:53:38.145224 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145210 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 05 11:53:38.145366 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145353 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 05 11:53:38.145446 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145426 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 05 11:53:38.145536 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145519 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 05 11:53:38.145613 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.145538 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c9qhj\"" Jul 05 11:53:38.156395 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.156374 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rk9gx" Jul 05 11:53:38.162595 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.162577 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rk9gx" Jul 05 11:53:38.224359 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.224328 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 05 11:53:38.235693 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235671 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-device-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.235855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j267d\" (UniqueName: \"kubernetes.io/projected/43583d36-b10f-4976-8829-03f8a1b1b7ea-kube-api-access-j267d\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.235855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab7a494-9ef9-4bb2-b923-43004cfff79f-host\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.235855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrzhs\" (UniqueName: \"kubernetes.io/projected/8ab7a494-9ef9-4bb2-b923-43004cfff79f-kube-api-access-hrzhs\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.235855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-sys\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.235855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235844 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-device-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235918 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-systemd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.235979 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d2td\" (UniqueName: \"kubernetes.io/projected/d0bfec19-86cb-4029-8270-4c50a93a4ef6-kube-api-access-8d2td\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236005 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-netns\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-tuned\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.236054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-netd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236103 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e98aea-c00e-4411-a3f1-f4d332dd5959-konnectivity-ca\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236212 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-k8s-cni-cncf-io\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236248 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-kubelet\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-conf-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236292 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qw6c\" (UniqueName: \"kubernetes.io/projected/ec20056c-246c-4228-be7c-06b6de585a85-kube-api-access-8qw6c\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-tmp\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnrnk\" (UniqueName: \"kubernetes.io/projected/75671e4c-023d-474c-9d77-fe29ca9b9044-kube-api-access-gnrnk\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-etc-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236442 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-ovn\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-bin\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236496 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236521 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-env-overrides\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236553 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-cnibin\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.236598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236580 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-sys-fs\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236635 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-sys-fs\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-os-release\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-script-lib\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236682 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-system-cni-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236698 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e98aea-c00e-4411-a3f1-f4d332dd5959-konnectivity-ca\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236736 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-system-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236791 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-lib-modules\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49sws\" (UniqueName: \"kubernetes.io/projected/c5da800a-b051-4182-92ae-40b4d44daeaf-kube-api-access-49sws\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-multus-daemon-config\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.236981 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.236988 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.236993 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-kubelet-config\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da800a-b051-4182-92ae-40b4d44daeaf-tmp-dir\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237025 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-etc-selinux\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.237053 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:38.737033345 +0000 UTC m=+2.995243806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc2qj\" (UniqueName: \"kubernetes.io/projected/50cc9a30-33a9-4375-b546-af0b5832bcfb-kube-api-access-rc2qj\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-socket-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysconfig\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-systemd\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-log-socket\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltk4m\" (UniqueName: \"kubernetes.io/projected/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kube-api-access-ltk4m\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-socket-dir-parent\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237283 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-socket-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.237532 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237301 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-kubernetes\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-run\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-node-log\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5da800a-b051-4182-92ae-40b4d44daeaf-hosts-file\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-conf\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237440 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-host\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-kubelet\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237530 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-netns\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237586 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-etc-kubernetes\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237608 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-config\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237629 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e98aea-c00e-4411-a3f1-f4d332dd5959-agent-certs\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.238111 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab7a494-9ef9-4bb2-b923-43004cfff79f-serviceca\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237676 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-cnibin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237703 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-cni-binary-copy\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-var-lib-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237746 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43583d36-b10f-4976-8829-03f8a1b1b7ea-host-slash\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-systemd-units\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237817 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/43583d36-b10f-4976-8829-03f8a1b1b7ea-host-slash\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237837 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-dbus\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-registration-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2xg\" (UniqueName: \"kubernetes.io/projected/6f110573-9832-4310-93b6-1bc140a48214-kube-api-access-qd2xg\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237913 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-registration-dir\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-multus\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.238746 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovn-node-metrics-cert\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.237986 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238011 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238005 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43583d36-b10f-4976-8829-03f8a1b1b7ea-iptables-alerter-script\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238156 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-bin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238192 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-multus-certs\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-var-lib-kubelet\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-slash\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-hostroot\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-modprobe-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-os-release\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.239238 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.238487 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/43583d36-b10f-4976-8829-03f8a1b1b7ea-iptables-alerter-script\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.241842 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.241822 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e98aea-c00e-4411-a3f1-f4d332dd5959-agent-certs\") pod \"konnectivity-agent-jvkdp\" (UID: \"e3e98aea-c00e-4411-a3f1-f4d332dd5959\") " pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.246426 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.246124 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:38.246426 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.246150 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:38.246426 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.246163 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:38.246426 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.246261 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:53:38.74624429 +0000 UTC m=+3.004454763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:38.246690 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.246576 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j267d\" (UniqueName: \"kubernetes.io/projected/43583d36-b10f-4976-8829-03f8a1b1b7ea-kube-api-access-j267d\") pod \"iptables-alerter-kstgl\" (UID: \"43583d36-b10f-4976-8829-03f8a1b1b7ea\") " pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.247685 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.247663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltk4m\" (UniqueName: \"kubernetes.io/projected/598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a-kube-api-access-ltk4m\") pod \"aws-ebs-csi-driver-node-968xm\" (UID: \"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.248142 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.248120 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2xg\" (UniqueName: \"kubernetes.io/projected/6f110573-9832-4310-93b6-1bc140a48214-kube-api-access-qd2xg\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.287624 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.287599 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:38.339613 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339579 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrzhs\" (UniqueName: \"kubernetes.io/projected/8ab7a494-9ef9-4bb2-b923-43004cfff79f-kube-api-access-hrzhs\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339628 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-sys\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-systemd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339700 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d2td\" (UniqueName: \"kubernetes.io/projected/d0bfec19-86cb-4029-8270-4c50a93a4ef6-kube-api-access-8d2td\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-netns\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-systemd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339803 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-netns\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339746 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-sys\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339803 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.339831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339749 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-tuned\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339885 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-netd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339920 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-k8s-cni-cncf-io\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339961 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.339984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-kubelet\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-conf-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340019 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-k8s-cni-cncf-io\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340031 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qw6c\" (UniqueName: \"kubernetes.io/projected/ec20056c-246c-4228-be7c-06b6de585a85-kube-api-access-8qw6c\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-tmp\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340064 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-kubelet\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340075 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnrnk\" (UniqueName: \"kubernetes.io/projected/75671e4c-023d-474c-9d77-fe29ca9b9044-kube-api-access-gnrnk\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-etc-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340103 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-netd\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.340347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-ovn\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-bin\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-env-overrides\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340199 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340252 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-etc-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340259 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-cnibin\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-run-ovn\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340302 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-cnibin\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340335 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-cni-bin\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-os-release\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340354 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-script-lib\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340372 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-system-cni-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.341022 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340408 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-system-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-lib-modules\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340457 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-system-cni-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340462 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-os-release\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340489 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-system-cni-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340541 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-conf-dir\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340831 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-env-overrides\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49sws\" (UniqueName: \"kubernetes.io/projected/c5da800a-b051-4182-92ae-40b4d44daeaf-kube-api-access-49sws\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340874 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-lib-modules\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-multus-daemon-config\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340906 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-kubelet-config\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.340959 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-kubelet-config\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341005 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-script-lib\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341034 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da800a-b051-4182-92ae-40b4d44daeaf-tmp-dir\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.341734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341062 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc2qj\" (UniqueName: \"kubernetes.io/projected/50cc9a30-33a9-4375-b546-af0b5832bcfb-kube-api-access-rc2qj\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341061 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341112 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysconfig\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341149 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-systemd\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-log-socket\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-socket-dir-parent\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341227 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysconfig\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-kubernetes\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341261 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-log-socket\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-run\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-node-log\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341282 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-kubernetes\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5da800a-b051-4182-92ae-40b4d44daeaf-hosts-file\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341312 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-run\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341336 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da800a-b051-4182-92ae-40b4d44daeaf-tmp-dir\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.342453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-conf\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341359 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-systemd\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341373 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-host\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-multus-socket-dir-parent\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341398 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341403 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5da800a-b051-4182-92ae-40b4d44daeaf-hosts-file\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-kubelet\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341428 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-node-log\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341447 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-host\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341458 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-multus-daemon-config\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341479 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-kubelet\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-netns\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341492 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-run-netns\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.341513 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341541 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-etc-kubernetes\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341515 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-etc-kubernetes\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341513 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-conf\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.341565 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:53:38.84154917 +0000 UTC m=+3.099759623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-config\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab7a494-9ef9-4bb2-b923-43004cfff79f-serviceca\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-cnibin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341720 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-cni-binary-copy\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-var-lib-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341812 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-systemd-units\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341835 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-dbus\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341863 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-multus\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341868 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-cnibin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovn-node-metrics-cert\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.343923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341918 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-systemd-units\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341960 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-var-lib-openvswitch\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341967 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-bin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.341999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-multus-certs\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342020 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-var-lib-kubelet\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-slash\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-hostroot\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342089 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-modprobe-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-os-release\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab7a494-9ef9-4bb2-b923-43004cfff79f-host\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342177 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342210 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab7a494-9ef9-4bb2-b923-43004cfff79f-serviceca\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342228 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab7a494-9ef9-4bb2-b923-43004cfff79f-host\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.344572 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342202 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovnkube-config\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342272 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73b426e4-95af-454b-a6d5-50e35d34a4fa-dbus\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342281 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0bfec19-86cb-4029-8270-4c50a93a4ef6-host-slash\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342269 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-bin\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-run-multus-certs\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342339 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-var-lib-kubelet\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342346 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-modprobe-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342363 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-hostroot\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342374 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ec20056c-246c-4228-be7c-06b6de585a85-host-var-lib-cni-multus\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342456 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-os-release\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342504 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc9a30-33a9-4375-b546-af0b5832bcfb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342629 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec20056c-246c-4228-be7c-06b6de585a85-cni-binary-copy\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342630 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-tuned\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342818 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75671e4c-023d-474c-9d77-fe29ca9b9044-tmp\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.342830 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc9a30-33a9-4375-b546-af0b5832bcfb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.345233 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.344353 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0bfec19-86cb-4029-8270-4c50a93a4ef6-ovn-node-metrics-cert\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.345855 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.345043 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75671e4c-023d-474c-9d77-fe29ca9b9044-etc-sysctl-d\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.349201 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.349175 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d2td\" (UniqueName: \"kubernetes.io/projected/d0bfec19-86cb-4029-8270-4c50a93a4ef6-kube-api-access-8d2td\") pod \"ovnkube-node-djdvv\" (UID: \"d0bfec19-86cb-4029-8270-4c50a93a4ef6\") " pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.349310 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.349275 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnrnk\" (UniqueName: \"kubernetes.io/projected/75671e4c-023d-474c-9d77-fe29ca9b9044-kube-api-access-gnrnk\") pod \"tuned-jkkph\" (UID: \"75671e4c-023d-474c-9d77-fe29ca9b9044\") " pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.349697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.349665 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrzhs\" (UniqueName: \"kubernetes.io/projected/8ab7a494-9ef9-4bb2-b923-43004cfff79f-kube-api-access-hrzhs\") pod \"node-ca-x7tsl\" (UID: \"8ab7a494-9ef9-4bb2-b923-43004cfff79f\") " pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.350562 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.350510 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qw6c\" (UniqueName: \"kubernetes.io/projected/ec20056c-246c-4228-be7c-06b6de585a85-kube-api-access-8qw6c\") pod \"multus-vjs65\" (UID: \"ec20056c-246c-4228-be7c-06b6de585a85\") " pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.350867 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.350845 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49sws\" (UniqueName: \"kubernetes.io/projected/c5da800a-b051-4182-92ae-40b4d44daeaf-kube-api-access-49sws\") pod \"node-resolver-f6l9g\" (UID: \"c5da800a-b051-4182-92ae-40b4d44daeaf\") " pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.351084 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.351064 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc2qj\" (UniqueName: \"kubernetes.io/projected/50cc9a30-33a9-4375-b546-af0b5832bcfb-kube-api-access-rc2qj\") pod \"multus-additional-cni-plugins-wjljl\" (UID: \"50cc9a30-33a9-4375-b546-af0b5832bcfb\") " pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.430162 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.430130 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" Jul 05 11:53:38.437113 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.437091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:53:38.447639 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.447619 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kstgl" Jul 05 11:53:38.455182 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.455162 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x7tsl" Jul 05 11:53:38.461607 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.461589 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f6l9g" Jul 05 11:53:38.471072 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.471056 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjljl" Jul 05 11:53:38.478521 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.478504 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vjs65" Jul 05 11:53:38.486990 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.486973 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jkkph" Jul 05 11:53:38.492529 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.492513 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:53:38.522449 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.522423 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 05 11:53:38.745787 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.745695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:38.745935 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.745850 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:38.745935 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.745924 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:39.74590205 +0000 UTC m=+4.004112494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:38.846263 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.846227 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:38.846427 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:38.846281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:38.846427 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846374 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:38.846427 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846388 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:38.846427 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846400 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:38.846427 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846409 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:38.846593 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846438 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:53:39.846423893 +0000 UTC m=+4.104634336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:38.846593 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:38.846455 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:53:39.846447368 +0000 UTC m=+4.104657806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:39.166530 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.166421 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 11:48:38 +0000 UTC" deadline="2027-12-29 09:51:07.667084861 +0000 UTC" Jul 05 11:53:39.166530 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.166471 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13005h57m28.500616905s" Jul 05 11:53:39.239013 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.238968 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:39.239570 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.239104 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:39.304840 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.304812 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab7a494_9ef9_4bb2_b923_43004cfff79f.slice/crio-e582e52c5cea49b3135b48f5a084c507eb5dca8bc816ea315c248e75118c79e8 WatchSource:0}: Error finding container e582e52c5cea49b3135b48f5a084c507eb5dca8bc816ea315c248e75118c79e8: Status 404 returned error can't find the container with id e582e52c5cea49b3135b48f5a084c507eb5dca8bc816ea315c248e75118c79e8 Jul 05 11:53:39.307561 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.307519 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cc9a30_33a9_4375_b546_af0b5832bcfb.slice/crio-4dd5fbf074998cbe76d2ef5c5a87013d88bf326374fef90331cc2e101c1c4e05 WatchSource:0}: Error finding container 4dd5fbf074998cbe76d2ef5c5a87013d88bf326374fef90331cc2e101c1c4e05: Status 404 returned error can't find the container with id 4dd5fbf074998cbe76d2ef5c5a87013d88bf326374fef90331cc2e101c1c4e05 Jul 05 11:53:39.310515 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.310493 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5da800a_b051_4182_92ae_40b4d44daeaf.slice/crio-f789038c6154500de3702858cd7b91df8e24d3e5bbf4554b957ea1e392860660 WatchSource:0}: Error finding container f789038c6154500de3702858cd7b91df8e24d3e5bbf4554b957ea1e392860660: Status 404 returned error can't find the container with id f789038c6154500de3702858cd7b91df8e24d3e5bbf4554b957ea1e392860660 Jul 05 11:53:39.311223 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.311182 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598b9895_cbd1_44e9_9ef2_d5dfd8c9fd4a.slice/crio-e049f868bda74f86fa875e1c6ccc89bbf5f44418c1aabebbbb6b137882484ff4 WatchSource:0}: Error finding container e049f868bda74f86fa875e1c6ccc89bbf5f44418c1aabebbbb6b137882484ff4: Status 404 returned error can't find the container with id e049f868bda74f86fa875e1c6ccc89bbf5f44418c1aabebbbb6b137882484ff4 Jul 05 11:53:39.312044 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.311985 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75671e4c_023d_474c_9d77_fe29ca9b9044.slice/crio-22a70e5f1691e4578ec04eafadee311791f95d29d13dbe3a03481c8d73bf2aa3 WatchSource:0}: Error finding container 22a70e5f1691e4578ec04eafadee311791f95d29d13dbe3a03481c8d73bf2aa3: Status 404 returned error can't find the container with id 22a70e5f1691e4578ec04eafadee311791f95d29d13dbe3a03481c8d73bf2aa3 Jul 05 11:53:39.312901 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.312820 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bfec19_86cb_4029_8270_4c50a93a4ef6.slice/crio-04d873ad6bca3cc1d7729db19786d29421248c4e95bc8fe05d3e8c1e654171c9 WatchSource:0}: Error finding container 04d873ad6bca3cc1d7729db19786d29421248c4e95bc8fe05d3e8c1e654171c9: Status 404 returned error can't find the container with id 04d873ad6bca3cc1d7729db19786d29421248c4e95bc8fe05d3e8c1e654171c9 Jul 05 11:53:39.315503 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.315189 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43583d36_b10f_4976_8829_03f8a1b1b7ea.slice/crio-518e8e8bfaaefd9b58bc9a558af1eec0658f04c45a219b6a52267afc76dd0deb WatchSource:0}: Error finding container 518e8e8bfaaefd9b58bc9a558af1eec0658f04c45a219b6a52267afc76dd0deb: Status 404 returned error can't find the container with id 518e8e8bfaaefd9b58bc9a558af1eec0658f04c45a219b6a52267afc76dd0deb Jul 05 11:53:39.316683 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.316661 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec20056c_246c_4228_be7c_06b6de585a85.slice/crio-1ea5a57ddc3267b9051eed825c4d1f200c775efa3c2b0b63293aba41810d2f45 WatchSource:0}: Error finding container 1ea5a57ddc3267b9051eed825c4d1f200c775efa3c2b0b63293aba41810d2f45: Status 404 returned error can't find the container with id 1ea5a57ddc3267b9051eed825c4d1f200c775efa3c2b0b63293aba41810d2f45 Jul 05 11:53:39.317022 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:53:39.316985 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e98aea_c00e_4411_a3f1_f4d332dd5959.slice/crio-7bdd28d2ac0d3130428613b3f7778b680fc6bb99d678bd3551a8b7e9ebe35545 WatchSource:0}: Error finding container 7bdd28d2ac0d3130428613b3f7778b680fc6bb99d678bd3551a8b7e9ebe35545: Status 404 returned error can't find the container with id 7bdd28d2ac0d3130428613b3f7778b680fc6bb99d678bd3551a8b7e9ebe35545 Jul 05 11:53:39.755615 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.753345 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:39.757932 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.757903 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:39.758059 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.758000 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:41.757978568 +0000 UTC m=+6.016189009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:39.854653 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.854611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:39.854841 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:39.854698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:39.854841 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.854835 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:39.854964 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.854897 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:53:41.854878968 +0000 UTC m=+6.113089416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:39.855027 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.854983 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:39.855027 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.854999 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:39.855027 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.855013 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:39.855173 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:39.855046 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:53:41.855035158 +0000 UTC m=+6.113245596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:40.167701 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.167557 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-04 11:48:38 +0000 UTC" deadline="2028-03-31 18:19:46.73970155 +0000 UTC" Jul 05 11:53:40.167701 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.167619 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15246h26m6.572087092s" Jul 05 11:53:40.241397 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.241364 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:40.241995 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:40.241506 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:40.241995 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.241915 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:40.242128 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:40.242016 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:40.274262 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.274230 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f6l9g" event={"ID":"c5da800a-b051-4182-92ae-40b4d44daeaf","Type":"ContainerStarted","Data":"f789038c6154500de3702858cd7b91df8e24d3e5bbf4554b957ea1e392860660"} Jul 05 11:53:40.293528 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.293445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerStarted","Data":"4dd5fbf074998cbe76d2ef5c5a87013d88bf326374fef90331cc2e101c1c4e05"} Jul 05 11:53:40.303310 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.303218 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x7tsl" event={"ID":"8ab7a494-9ef9-4bb2-b923-43004cfff79f","Type":"ContainerStarted","Data":"e582e52c5cea49b3135b48f5a084c507eb5dca8bc816ea315c248e75118c79e8"} Jul 05 11:53:40.308860 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.308201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" event={"ID":"71665d816167a69ba4f61ac61f31e766","Type":"ContainerStarted","Data":"e4b2b95b8586295b485bedf45d31cb466fa96988c6691944bfbdb8632239ab54"} Jul 05 11:53:40.323229 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.323100 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" event={"ID":"71246e965b265fa9911eff2ab0b350b8","Type":"ContainerDied","Data":"416457cd4ca9383d6cae0f83a0a32cd8cc790fb906ef2c9fc3b8df07bdf1d85d"} Jul 05 11:53:40.323229 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.323063 2574 generic.go:356] "Generic (PLEG): container finished" podID="71246e965b265fa9911eff2ab0b350b8" containerID="416457cd4ca9383d6cae0f83a0a32cd8cc790fb906ef2c9fc3b8df07bdf1d85d" exitCode=0 Jul 05 11:53:40.328937 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.328893 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jvkdp" event={"ID":"e3e98aea-c00e-4411-a3f1-f4d332dd5959","Type":"ContainerStarted","Data":"7bdd28d2ac0d3130428613b3f7778b680fc6bb99d678bd3551a8b7e9ebe35545"} Jul 05 11:53:40.336096 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.335954 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kstgl" event={"ID":"43583d36-b10f-4976-8829-03f8a1b1b7ea","Type":"ContainerStarted","Data":"518e8e8bfaaefd9b58bc9a558af1eec0658f04c45a219b6a52267afc76dd0deb"} Jul 05 11:53:40.338016 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.337960 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-202.ec2.internal" podStartSLOduration=3.33794403 podStartE2EDuration="3.33794403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:40.323123446 +0000 UTC m=+4.581333929" watchObservedRunningTime="2026-07-05 11:53:40.33794403 +0000 UTC m=+4.596154494" Jul 05 11:53:40.357391 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.357364 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"04d873ad6bca3cc1d7729db19786d29421248c4e95bc8fe05d3e8c1e654171c9"} Jul 05 11:53:40.375455 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.372085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkkph" event={"ID":"75671e4c-023d-474c-9d77-fe29ca9b9044","Type":"ContainerStarted","Data":"22a70e5f1691e4578ec04eafadee311791f95d29d13dbe3a03481c8d73bf2aa3"} Jul 05 11:53:40.380078 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.379985 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" event={"ID":"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a","Type":"ContainerStarted","Data":"e049f868bda74f86fa875e1c6ccc89bbf5f44418c1aabebbbb6b137882484ff4"} Jul 05 11:53:40.387605 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:40.387578 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vjs65" event={"ID":"ec20056c-246c-4228-be7c-06b6de585a85","Type":"ContainerStarted","Data":"1ea5a57ddc3267b9051eed825c4d1f200c775efa3c2b0b63293aba41810d2f45"} Jul 05 11:53:41.241117 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:41.240975 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:41.241117 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.241106 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:41.408859 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:41.408822 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" event={"ID":"71246e965b265fa9911eff2ab0b350b8","Type":"ContainerStarted","Data":"c6f2d9b61f07bce18bb57b3859d2bbf446d13e6d251ab8a7aa1464b014fabf16"} Jul 05 11:53:41.768205 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:41.768169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:41.768389 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.768311 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:41.768389 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.768382 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:45.768364164 +0000 UTC m=+10.026574606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:41.869250 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:41.869330 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.869464 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.869537 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:53:45.869520563 +0000 UTC m=+10.127731025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.869991 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.870012 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.870025 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:41.870131 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:41.870071 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:53:45.870054213 +0000 UTC m=+10.128264665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:42.241509 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:42.241481 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:42.241698 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:42.241616 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:42.241945 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:42.241924 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:42.242081 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:42.242058 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:43.238587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:43.238552 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:43.239054 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:43.238663 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:44.239631 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:44.239597 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:44.240071 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:44.239726 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:44.240071 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:44.240048 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:44.241422 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:44.240157 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:45.238712 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:45.238631 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:45.238921 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.238778 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:45.803411 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:45.802841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:45.803411 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.802992 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:45.803411 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.803052 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:53:53.803033681 +0000 UTC m=+18.061244122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:45.903421 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:45.903378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:45.903588 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:45.903459 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:45.903661 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.903591 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:45.903661 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.903651 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:53:53.903633753 +0000 UTC m=+18.161844194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:45.904034 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.904007 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:45.904090 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.904044 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:45.904090 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.904059 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:45.904180 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:45.904115 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:53:53.90409907 +0000 UTC m=+18.162309527 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:46.242463 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:46.242422 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:46.242684 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:46.242577 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:46.242743 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:46.242700 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:46.243061 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:46.243031 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:47.238984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:47.238945 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:47.239493 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:47.239073 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:48.238424 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:48.238349 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:48.238579 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:48.238480 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:48.238579 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:48.238533 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:48.238685 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:48.238657 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:49.238781 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:49.238737 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:49.239214 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:49.238851 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:50.238391 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:50.238356 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:50.238520 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:50.238480 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:50.238520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:50.238512 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:50.238625 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:50.238570 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:51.238214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:51.238176 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:51.238641 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:51.238323 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:52.238960 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:52.238923 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:52.239431 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:52.239070 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:52.239431 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:52.239125 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:52.239431 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:52.239231 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:53.238680 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:53.238643 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:53.238903 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.238754 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:53.862741 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:53.862699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:53.863110 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.862867 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:53.863110 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.862937 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:09.862919529 +0000 UTC m=+34.121129966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:53:53.963672 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:53.963644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:53.963795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:53.963692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:53.963834 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963800 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:53.963834 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963819 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:53:53.963887 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963841 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:53:53.963887 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963847 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:54:09.963835321 +0000 UTC m=+34.222045759 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:53:53.963887 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963851 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:53.963986 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:53.963889 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:54:09.96387827 +0000 UTC m=+34.222088708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:53:54.239237 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:54.238724 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:54.239237 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:54.238781 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:54.239237 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:54.238888 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:54.239237 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:54.239039 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:55.238414 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:55.238383 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:55.238847 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:55.238494 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:56.241260 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:56.241231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:56.241671 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:56.241353 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:56.241671 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:56.241423 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:56.241671 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:56.241558 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:57.238436 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:57.238399 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:57.238618 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:57.238532 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:58.241249 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:58.241225 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:53:58.241553 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:58.241338 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:53:58.241553 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:58.241381 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:53:58.241553 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:58.241476 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:53:59.239228 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.238920 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:53:59.239390 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:53:59.239298 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:53:59.487727 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.487690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vjs65" event={"ID":"ec20056c-246c-4228-be7c-06b6de585a85","Type":"ContainerStarted","Data":"8ed80a148bfe09115b55c4487d15c61d0a11475f85ee26903427d2fd0fdb9fea"} Jul 05 11:53:59.489066 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.489040 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f6l9g" event={"ID":"c5da800a-b051-4182-92ae-40b4d44daeaf","Type":"ContainerStarted","Data":"4a9f44bd442b6e14d9c7ea0650b2837d9aa0497271d296477891b8914f26d20f"} Jul 05 11:53:59.490429 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.490406 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="9f20ddf203b7e7078979abb1e45bdf7d5b5df5641cead3e1c564b320e1028a2f" exitCode=0 Jul 05 11:53:59.490597 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.490487 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"9f20ddf203b7e7078979abb1e45bdf7d5b5df5641cead3e1c564b320e1028a2f"} Jul 05 11:53:59.491954 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.491935 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x7tsl" event={"ID":"8ab7a494-9ef9-4bb2-b923-43004cfff79f","Type":"ContainerStarted","Data":"395b41a1b3a486e4d3957ac8072eabe5d4d35f19009aaaa426e938921905a8ae"} Jul 05 11:53:59.493341 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.493318 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jvkdp" event={"ID":"e3e98aea-c00e-4411-a3f1-f4d332dd5959","Type":"ContainerStarted","Data":"9d98942252a6e543abbf340f35b3e205c6c9641e0d0a036afda08a5a3c638280"} Jul 05 11:53:59.496252 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496234 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"efa1cd7623b1b2469ea937b78f4aca1408f4aa376680bd012efa703359154e81"} Jul 05 11:53:59.496344 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"7bc7a8d66b2003a4cf6c4473c31bd33085dfe33d72ea35e43252aafd065a7ba8"} Jul 05 11:53:59.496395 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496342 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"4d4de33d0db1aae2f7310bed973dbc2ed26a9fca5d78e91228aa160d2fc3684b"} Jul 05 11:53:59.496430 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496392 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"a2fea6a4737f22e5ae7326ca8094b984c04953c987eb788deb8ef38e84a57e98"} Jul 05 11:53:59.496478 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496439 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"d17de0d3d2c747bcd31eabf66428169bfc85e4b66da89638523fd513f44812c7"} Jul 05 11:53:59.496478 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.496473 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"a347e425adf0660987960a6f8a8e0dc79897c0cf4965ab91b24707ea2a57506d"} Jul 05 11:53:59.497589 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.497560 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jkkph" event={"ID":"75671e4c-023d-474c-9d77-fe29ca9b9044","Type":"ContainerStarted","Data":"3b67e527ba85387e06e7df79ebc66b27bcbc8486828165bedd05dd93e0a6636e"} Jul 05 11:53:59.498868 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.498851 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" event={"ID":"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a","Type":"ContainerStarted","Data":"14d1ffea029cf6b5bc1e0182539d41e0f8f5278c6de2c73cdafefcdcb4988866"} Jul 05 11:53:59.505483 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.503603 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-202.ec2.internal" podStartSLOduration=22.503585693 podStartE2EDuration="22.503585693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:41.424514462 +0000 UTC m=+5.682724934" watchObservedRunningTime="2026-07-05 11:53:59.503585693 +0000 UTC m=+23.761796181" Jul 05 11:53:59.505483 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.504264 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vjs65" podStartSLOduration=4.40989109 podStartE2EDuration="23.504253863s" totalImagesPullingTime="19.094362773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:59.503364034 +0000 UTC m=+23.761574494" watchObservedRunningTime="2026-07-05 11:53:59.504253863 +0000 UTC m=+23.762464322" Jul 05 11:53:59.530854 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.530816 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jkkph" podStartSLOduration=4.492602821 podStartE2EDuration="23.53080451s" totalImagesPullingTime="19.038201689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:59.518282059 +0000 UTC m=+23.776492531" watchObservedRunningTime="2026-07-05 11:53:59.53080451 +0000 UTC m=+23.789014969" Jul 05 11:53:59.531150 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.531123 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jvkdp" podStartSLOduration=4.524314433 podStartE2EDuration="23.531113817s" totalImagesPullingTime="19.006799384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:59.530461409 +0000 UTC m=+23.788671868" watchObservedRunningTime="2026-07-05 11:53:59.531113817 +0000 UTC m=+23.789324278" Jul 05 11:53:59.542988 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.542951 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x7tsl" podStartSLOduration=4.494460871 podStartE2EDuration="23.542939415s" totalImagesPullingTime="19.048478544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:59.542558732 +0000 UTC m=+23.800769192" watchObservedRunningTime="2026-07-05 11:53:59.542939415 +0000 UTC m=+23.801149924" Jul 05 11:53:59.577584 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.577548 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f6l9g" podStartSLOduration=4.53438948 podStartE2EDuration="23.57753582s" totalImagesPullingTime="19.04314634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:53:59.556262832 +0000 UTC m=+23.814473288" watchObservedRunningTime="2026-07-05 11:53:59.57753582 +0000 UTC m=+23.835746279" Jul 05 11:53:59.858978 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:53:59.858954 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 05 11:54:00.184019 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.183913 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-05T11:53:59.858973103Z","UUID":"110bd6df-eb3b-431e-9f04-5a62bd8c774a","Handler":null,"Name":"","Endpoint":""} Jul 05 11:54:00.185615 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.185595 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 05 11:54:00.185778 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.185633 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 05 11:54:00.238587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.238561 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:00.238754 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.238585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:00.238754 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:00.238692 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:00.238891 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:00.238812 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:00.503028 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.502934 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kstgl" event={"ID":"43583d36-b10f-4976-8829-03f8a1b1b7ea","Type":"ContainerStarted","Data":"a5a68084c250ff8f7684ed0cb3d87066b6c59177c5388dd749df76afe214395c"} Jul 05 11:54:00.505150 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.505110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" event={"ID":"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a","Type":"ContainerStarted","Data":"330a7844e9966bb8f39527c539ec8e04d137f93be554c9a6fbc9025d7ba0880d"} Jul 05 11:54:00.516483 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:00.516444 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kstgl" podStartSLOduration=5.508101958 podStartE2EDuration="24.516430976s" totalImagesPullingTime="19.008329018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:00.515914236 +0000 UTC m=+24.774124698" watchObservedRunningTime="2026-07-05 11:54:00.516430976 +0000 UTC m=+24.774641436" Jul 05 11:54:01.238722 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.238689 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:01.238937 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:01.238827 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:01.487983 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.487620 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:54:01.488331 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.488309 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:54:01.511049 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.510969 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"6bdbc9f38a4d98c853aaefaa712139fa325922d6d834ce77a6f3cfc74f26c758"} Jul 05 11:54:01.512992 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.512960 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" event={"ID":"598b9895-cbd1-44e9-9ef2-d5dfd8c9fd4a","Type":"ContainerStarted","Data":"19cd4c5f56a8309cbb8d4000fbf9ef9c7601bf3081657cbc34b86a9b08390cb6"} Jul 05 11:54:01.528450 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:01.528410 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-968xm" podStartSLOduration=4.320299758 podStartE2EDuration="25.528394919s" totalImagesPullingTime="21.208095161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:01.528094274 +0000 UTC m=+25.786304735" watchObservedRunningTime="2026-07-05 11:54:01.528394919 +0000 UTC m=+25.786605376" Jul 05 11:54:02.239240 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:02.239207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:02.239552 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:02.239248 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:02.239552 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:02.239335 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:02.239552 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:02.239455 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:02.515749 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:02.515669 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 05 11:54:03.239155 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:03.239108 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:03.239331 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:03.239239 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:04.125251 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:04.125219 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:54:04.125865 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:04.125344 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 05 11:54:04.125865 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:04.125779 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jvkdp" Jul 05 11:54:04.238384 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:04.238286 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:04.238523 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:04.238422 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:04.238523 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:04.238477 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:04.238610 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:04.238577 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:05.238926 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.238728 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:05.239457 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:05.239000 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:05.524782 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.524680 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" event={"ID":"d0bfec19-86cb-4029-8270-4c50a93a4ef6","Type":"ContainerStarted","Data":"d1368d75fc870fccbdc8576bb84dfb5cc96be9e498123c9145d90b4010817e69"} Jul 05 11:54:05.525011 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.524873 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:05.525011 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.524945 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:05.525011 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.524996 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:05.526711 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.526687 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="ec2fd39f3327b09f5197df12c83a3305bfe7b2dea8b55a492769300749101545" exitCode=0 Jul 05 11:54:05.526833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.526722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"ec2fd39f3327b09f5197df12c83a3305bfe7b2dea8b55a492769300749101545"} Jul 05 11:54:05.541132 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.541108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:05.541247 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.541213 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:05.550704 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:05.550665 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" podStartSLOduration=10.441731575 podStartE2EDuration="29.550651141s" totalImagesPullingTime="19.108919566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:05.549083922 +0000 UTC m=+29.807294398" watchObservedRunningTime="2026-07-05 11:54:05.550651141 +0000 UTC m=+29.808861600" Jul 05 11:54:06.239784 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.239737 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:06.240319 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:06.239879 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:06.240319 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.239894 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:06.240319 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:06.240020 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:06.516557 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.516474 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29x85"] Jul 05 11:54:06.519618 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.519591 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsfb7"] Jul 05 11:54:06.519773 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.519708 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:06.519848 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:06.519831 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:06.520171 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.520127 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-prgqv"] Jul 05 11:54:06.530053 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.529395 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:06.530053 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:06.529485 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:06.530053 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:06.529897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:06.530053 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:06.530042 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:07.533739 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:07.533711 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="e728fda9cce7b41b37e250ce0515e0a6fc8423097415eb6d14e2c2ff20a30d96" exitCode=0 Jul 05 11:54:07.534154 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:07.533789 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"e728fda9cce7b41b37e250ce0515e0a6fc8423097415eb6d14e2c2ff20a30d96"} Jul 05 11:54:07.883792 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:07.883711 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f6l9g_c5da800a-b051-4182-92ae-40b4d44daeaf/dns-node-resolver/0.log" Jul 05 11:54:08.241289 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:08.241264 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:08.241455 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:08.241351 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:08.241455 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:08.241355 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:08.241455 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:08.241380 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:08.241612 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:08.241459 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:08.241612 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:08.241556 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:08.538566 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:08.538537 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerStarted","Data":"467687ff6431afb39535e4914febfb9d0fcf1470b42e16ef9038e498c62030ea"} Jul 05 11:54:08.856546 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:08.856517 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x7tsl_8ab7a494-9ef9-4bb2-b923-43004cfff79f/node-ca/0.log" Jul 05 11:54:09.543492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:09.543413 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="467687ff6431afb39535e4914febfb9d0fcf1470b42e16ef9038e498c62030ea" exitCode=0 Jul 05 11:54:09.543492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:09.543451 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"467687ff6431afb39535e4914febfb9d0fcf1470b42e16ef9038e498c62030ea"} Jul 05 11:54:09.880989 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:09.880916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:09.881124 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.881059 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:54:09.881124 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.881112 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs podName:6f110573-9832-4310-93b6-1bc140a48214 nodeName:}" failed. No retries permitted until 2026-07-05 11:54:41.881098085 +0000 UTC m=+66.139308523 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs") pod "network-metrics-daemon-29x85" (UID: "6f110573-9832-4310-93b6-1bc140a48214") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 05 11:54:09.981832 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:09.981802 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:09.982006 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:09.981855 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:09.982006 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.981954 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 05 11:54:09.982006 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.981965 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 05 11:54:09.982006 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.981979 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 05 11:54:09.982006 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.981991 2574 projected.go:196] Error preparing data for projected volume kube-api-access-c7gwq for pod openshift-network-diagnostics/network-check-target-tsfb7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:54:09.982264 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.982016 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret podName:73b426e4-95af-454b-a6d5-50e35d34a4fa nodeName:}" failed. No retries permitted until 2026-07-05 11:54:41.981998962 +0000 UTC m=+66.240209409 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret") pod "global-pull-secret-syncer-prgqv" (UID: "73b426e4-95af-454b-a6d5-50e35d34a4fa") : object "kube-system"/"original-pull-secret" not registered Jul 05 11:54:09.982264 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:09.982033 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq podName:1358a8e0-1566-42e0-9b2d-5de59627c67f nodeName:}" failed. No retries permitted until 2026-07-05 11:54:41.982024542 +0000 UTC m=+66.240234994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c7gwq" (UniqueName: "kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq") pod "network-check-target-tsfb7" (UID: "1358a8e0-1566-42e0-9b2d-5de59627c67f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 05 11:54:10.242441 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:10.242414 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:10.242633 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:10.242417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:10.242633 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:10.242522 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:10.242633 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:10.242419 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:10.242633 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:10.242614 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:10.242827 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:10.242694 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:12.238873 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:12.238841 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:12.239467 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:12.238973 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:12.239467 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:12.238985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:12.239467 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:12.239057 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:12.239467 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:12.239163 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:12.239467 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:12.239249 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:14.241175 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:14.241146 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:14.241548 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:14.241246 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:14.241548 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:14.241270 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:14.241548 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:14.241261 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:14.241548 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:14.241354 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:14.241548 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:14.241435 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:14.557470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:14.557396 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="c127bab9cf960773dee33b51acb377c0a159374405f07993356cec8afcfc97ed" exitCode=0 Jul 05 11:54:14.557470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:14.557435 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"c127bab9cf960773dee33b51acb377c0a159374405f07993356cec8afcfc97ed"} Jul 05 11:54:15.562416 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:15.562228 2574 generic.go:356] "Generic (PLEG): container finished" podID="50cc9a30-33a9-4375-b546-af0b5832bcfb" containerID="270ba4519aa62eb40ddbf942ef55a32ab86a2971e728bd8317df7838596f38d4" exitCode=0 Jul 05 11:54:15.562820 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:15.562313 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerDied","Data":"270ba4519aa62eb40ddbf942ef55a32ab86a2971e728bd8317df7838596f38d4"} Jul 05 11:54:16.239375 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:16.239348 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:16.239526 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:16.239417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:16.239526 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:16.239431 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:16.239526 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:16.239496 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:16.239667 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:16.239574 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:16.239667 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:16.239638 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:16.567315 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:16.567281 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjljl" event={"ID":"50cc9a30-33a9-4375-b546-af0b5832bcfb","Type":"ContainerStarted","Data":"d3082f150053063039c8c587ff1676ebcfeefd81a1d74b2355c79bf7aec244e6"} Jul 05 11:54:16.592731 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:16.592690 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wjljl" podStartSLOduration=10.191194464 podStartE2EDuration="40.592677197s" totalImagesPullingTime="30.401482733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:16.591167248 +0000 UTC m=+40.849377708" watchObservedRunningTime="2026-07-05 11:54:16.592677197 +0000 UTC m=+40.850887658" Jul 05 11:54:18.238861 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:18.238825 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:18.238861 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:18.238861 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:18.239283 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:18.238823 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:18.239283 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:18.238941 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:18.239283 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:18.239008 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:18.239283 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:18.239076 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:20.238976 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:20.238942 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:20.238976 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:20.238971 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:20.239512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:20.238947 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:20.239512 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:20.239049 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:20.239512 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:20.239131 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:20.239512 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:20.239186 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:22.238857 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:22.238820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:22.239348 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:22.238820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:22.239348 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:22.238943 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:22.239348 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:22.238820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:22.239348 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:22.239030 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:22.239348 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:22.239085 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:24.239113 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:24.239077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:24.239500 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:24.239085 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:24.239500 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:24.239215 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:24.239500 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:24.239086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:24.239500 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:24.239266 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:24.239500 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:24.239344 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:26.239538 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:26.239499 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:26.239982 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:26.239606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:26.239982 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:26.239634 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29x85" podUID="6f110573-9832-4310-93b6-1bc140a48214" Jul 05 11:54:26.239982 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:26.239681 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsfb7" podUID="1358a8e0-1566-42e0-9b2d-5de59627c67f" Jul 05 11:54:26.239982 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:26.239734 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:26.239982 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:26.239857 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-prgqv" podUID="73b426e4-95af-454b-a6d5-50e35d34a4fa" Jul 05 11:54:27.110699 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.110673 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-202.ec2.internal" event="NodeReady" Jul 05 11:54:27.110846 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.110808 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 05 11:54:27.142625 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.142544 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-646c4846d9-bl7dd"] Jul 05 11:54:27.154612 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.154585 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n87rx"] Jul 05 11:54:27.154747 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.154692 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.158046 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.158008 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 05 11:54:27.158184 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.158080 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mbvkr\"" Jul 05 11:54:27.158184 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.158159 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 05 11:54:27.158310 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.158276 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 05 11:54:27.164907 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.164731 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 05 11:54:27.165849 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.165828 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cl6j9"] Jul 05 11:54:27.166487 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.166466 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.168999 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.168953 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gbwbs\"" Jul 05 11:54:27.169236 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.169215 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 05 11:54:27.169236 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.169227 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 05 11:54:27.180476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.180457 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646c4846d9-bl7dd"] Jul 05 11:54:27.180562 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.180485 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n87rx"] Jul 05 11:54:27.180562 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.180502 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cl6j9"] Jul 05 11:54:27.180562 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.180557 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.183156 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.183134 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 05 11:54:27.183156 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.183150 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 05 11:54:27.183272 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.183166 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 05 11:54:27.183467 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.183448 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bxxhf\"" Jul 05 11:54:27.183550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.183450 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 05 11:54:27.256776 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.256742 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xs6k6"] Jul 05 11:54:27.270734 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.270702 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xs6k6"] Jul 05 11:54:27.270887 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.270704 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.273360 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.273326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 05 11:54:27.273494 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.273380 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c4hs5\"" Jul 05 11:54:27.273494 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.273384 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 05 11:54:27.273494 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.273447 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 05 11:54:27.313069 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313037 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w76z\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-kube-api-access-6w76z\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.313196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-trusted-ca\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313133 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a8e5964-b285-4b87-b31d-6a165e12c20b-config-volume\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.313196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-bound-sa-token\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313372 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313196 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e06f685-a431-4af2-adfe-d13f498b1d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.313372 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndhpf\" (UniqueName: \"kubernetes.io/projected/9a8e5964-b285-4b87-b31d-6a165e12c20b-kube-api-access-ndhpf\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.313372 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313325 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/651e7d2b-fea9-419f-bd91-3b3ae586f620-ca-trust-extracted\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313372 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e06f685-a431-4af2-adfe-d13f498b1d26-crio-socket\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.313372 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e06f685-a431-4af2-adfe-d13f498b1d26-data-volume\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313388 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-image-registry-private-configuration\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313442 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crvhz\" (UniqueName: \"kubernetes.io/projected/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-api-access-crvhz\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313462 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a8e5964-b285-4b87-b31d-6a165e12c20b-metrics-tls\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a8e5964-b285-4b87-b31d-6a165e12c20b-tmp-dir\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313525 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-tls\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-certificates\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.313598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.313577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-installation-pull-secrets\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.414678 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414604 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crvhz\" (UniqueName: \"kubernetes.io/projected/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-api-access-crvhz\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.414678 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414636 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a8e5964-b285-4b87-b31d-6a165e12c20b-metrics-tls\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.414678 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a8e5964-b285-4b87-b31d-6a165e12c20b-tmp-dir\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.414678 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414669 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-tls\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415012 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-certificates\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415012 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-installation-pull-secrets\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415012 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414803 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w76z\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-kube-api-access-6w76z\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415012 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/567bbb08-f82e-4ce3-bf69-2c3e236990e2-cert\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.415012 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.414862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fdsq\" (UniqueName: \"kubernetes.io/projected/567bbb08-f82e-4ce3-bf69-2c3e236990e2-kube-api-access-2fdsq\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415068 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a8e5964-b285-4b87-b31d-6a165e12c20b-tmp-dir\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-trusted-ca\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415157 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a8e5964-b285-4b87-b31d-6a165e12c20b-config-volume\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-bound-sa-token\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415232 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e06f685-a431-4af2-adfe-d13f498b1d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndhpf\" (UniqueName: \"kubernetes.io/projected/9a8e5964-b285-4b87-b31d-6a165e12c20b-kube-api-access-ndhpf\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415300 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/651e7d2b-fea9-419f-bd91-3b3ae586f620-ca-trust-extracted\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e06f685-a431-4af2-adfe-d13f498b1d26-crio-socket\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415364 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e06f685-a431-4af2-adfe-d13f498b1d26-data-volume\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-image-registry-private-configuration\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415544 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415868 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415644 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e06f685-a431-4af2-adfe-d13f498b1d26-crio-socket\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.415868 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415667 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-certificates\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.415970 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.415939 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/651e7d2b-fea9-419f-bd91-3b3ae586f620-ca-trust-extracted\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.416081 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.416060 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/651e7d2b-fea9-419f-bd91-3b3ae586f620-trusted-ca\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.416299 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.416185 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e06f685-a431-4af2-adfe-d13f498b1d26-data-volume\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.416299 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.416216 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a8e5964-b285-4b87-b31d-6a165e12c20b-config-volume\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.419292 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.419262 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e06f685-a431-4af2-adfe-d13f498b1d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.419419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.419317 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-image-registry-private-configuration\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.419419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.419332 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a8e5964-b285-4b87-b31d-6a165e12c20b-metrics-tls\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.419419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.419319 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/651e7d2b-fea9-419f-bd91-3b3ae586f620-installation-pull-secrets\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.419419 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.419412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-registry-tls\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.422359 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.422336 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crvhz\" (UniqueName: \"kubernetes.io/projected/0e06f685-a431-4af2-adfe-d13f498b1d26-kube-api-access-crvhz\") pod \"insights-runtime-extractor-cl6j9\" (UID: \"0e06f685-a431-4af2-adfe-d13f498b1d26\") " pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.422748 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.422728 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndhpf\" (UniqueName: \"kubernetes.io/projected/9a8e5964-b285-4b87-b31d-6a165e12c20b-kube-api-access-ndhpf\") pod \"dns-default-n87rx\" (UID: \"9a8e5964-b285-4b87-b31d-6a165e12c20b\") " pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.422960 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.422945 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-bound-sa-token\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.423074 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.423053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w76z\" (UniqueName: \"kubernetes.io/projected/651e7d2b-fea9-419f-bd91-3b3ae586f620-kube-api-access-6w76z\") pod \"image-registry-646c4846d9-bl7dd\" (UID: \"651e7d2b-fea9-419f-bd91-3b3ae586f620\") " pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.468476 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.468448 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:27.476584 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.476554 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:27.488374 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.488362 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cl6j9" Jul 05 11:54:27.516462 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.516429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/567bbb08-f82e-4ce3-bf69-2c3e236990e2-cert\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.516588 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.516483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fdsq\" (UniqueName: \"kubernetes.io/projected/567bbb08-f82e-4ce3-bf69-2c3e236990e2-kube-api-access-2fdsq\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.524292 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.524259 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fdsq\" (UniqueName: \"kubernetes.io/projected/567bbb08-f82e-4ce3-bf69-2c3e236990e2-kube-api-access-2fdsq\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.529602 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.529534 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/567bbb08-f82e-4ce3-bf69-2c3e236990e2-cert\") pod \"ingress-canary-xs6k6\" (UID: \"567bbb08-f82e-4ce3-bf69-2c3e236990e2\") " pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.580058 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.579989 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xs6k6" Jul 05 11:54:27.680402 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.680326 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646c4846d9-bl7dd"] Jul 05 11:54:27.684475 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:27.684448 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod651e7d2b_fea9_419f_bd91_3b3ae586f620.slice/crio-0210fdcf58a571afe2231d64df3f2405a77f2905ae18291e5f20a5291bb69c97 WatchSource:0}: Error finding container 0210fdcf58a571afe2231d64df3f2405a77f2905ae18291e5f20a5291bb69c97: Status 404 returned error can't find the container with id 0210fdcf58a571afe2231d64df3f2405a77f2905ae18291e5f20a5291bb69c97 Jul 05 11:54:27.749710 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.749684 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xs6k6"] Jul 05 11:54:27.754806 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:27.754781 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567bbb08_f82e_4ce3_bf69_2c3e236990e2.slice/crio-71cef97328bab5ce174e556dcada6ca0258e2758b1748a70262029696fe6da58 WatchSource:0}: Error finding container 71cef97328bab5ce174e556dcada6ca0258e2758b1748a70262029696fe6da58: Status 404 returned error can't find the container with id 71cef97328bab5ce174e556dcada6ca0258e2758b1748a70262029696fe6da58 Jul 05 11:54:27.898412 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.898129 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cl6j9"] Jul 05 11:54:27.900794 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:27.900748 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e06f685_a431_4af2_adfe_d13f498b1d26.slice/crio-b8aeb4420b9c9cd46932eb3f788da4bed011c7a61ef79aa01ed1cb6232b7ad61 WatchSource:0}: Error finding container b8aeb4420b9c9cd46932eb3f788da4bed011c7a61ef79aa01ed1cb6232b7ad61: Status 404 returned error can't find the container with id b8aeb4420b9c9cd46932eb3f788da4bed011c7a61ef79aa01ed1cb6232b7ad61 Jul 05 11:54:27.901319 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:27.901284 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n87rx"] Jul 05 11:54:27.903821 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:27.903799 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8e5964_b285_4b87_b31d_6a165e12c20b.slice/crio-1b0eec4ebf257807f221f88b4ac0c814a1b4bad59da67a552dad16e39f6558c4 WatchSource:0}: Error finding container 1b0eec4ebf257807f221f88b4ac0c814a1b4bad59da67a552dad16e39f6558c4: Status 404 returned error can't find the container with id 1b0eec4ebf257807f221f88b4ac0c814a1b4bad59da67a552dad16e39f6558c4 Jul 05 11:54:28.238792 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.238746 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:28.238969 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.238889 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:28.239141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.238777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:28.242453 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.242418 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 11:54:28.243106 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.243080 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 11:54:28.243663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.243376 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-759jc\"" Jul 05 11:54:28.243663 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.243628 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dsspw\"" Jul 05 11:54:28.243899 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.243884 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 11:54:28.244001 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.243982 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 11:54:28.605019 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.604931 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cl6j9" event={"ID":"0e06f685-a431-4af2-adfe-d13f498b1d26","Type":"ContainerStarted","Data":"b8aeb4420b9c9cd46932eb3f788da4bed011c7a61ef79aa01ed1cb6232b7ad61"} Jul 05 11:54:28.606944 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.606914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n87rx" event={"ID":"9a8e5964-b285-4b87-b31d-6a165e12c20b","Type":"ContainerStarted","Data":"1b0eec4ebf257807f221f88b4ac0c814a1b4bad59da67a552dad16e39f6558c4"} Jul 05 11:54:28.609520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.609494 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" event={"ID":"651e7d2b-fea9-419f-bd91-3b3ae586f620","Type":"ContainerStarted","Data":"964035e0896fd7238042bf0732d897ac4afa82f88dc43ba33cb3708aa8cd27d3"} Jul 05 11:54:28.609624 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.609539 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" event={"ID":"651e7d2b-fea9-419f-bd91-3b3ae586f620","Type":"ContainerStarted","Data":"0210fdcf58a571afe2231d64df3f2405a77f2905ae18291e5f20a5291bb69c97"} Jul 05 11:54:28.609688 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.609662 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:28.611574 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.611533 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xs6k6" event={"ID":"567bbb08-f82e-4ce3-bf69-2c3e236990e2","Type":"ContainerStarted","Data":"71cef97328bab5ce174e556dcada6ca0258e2758b1748a70262029696fe6da58"} Jul 05 11:54:28.630043 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:28.629969 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" podStartSLOduration=2.629937769 podStartE2EDuration="2.629937769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:28.627586152 +0000 UTC m=+52.885796613" watchObservedRunningTime="2026-07-05 11:54:28.629937769 +0000 UTC m=+52.888148234" Jul 05 11:54:29.063589 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.063429 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-hkrq2"] Jul 05 11:54:29.079936 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.079137 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-hkrq2"] Jul 05 11:54:29.079936 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.079254 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:29.082721 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.082096 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 05 11:54:29.082721 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.082265 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-64hq2\"" Jul 05 11:54:29.082721 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.082315 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 05 11:54:29.129154 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.128970 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gxww\" (UniqueName: \"kubernetes.io/projected/2afbac7d-2575-439e-b7a5-f66063a6417a-kube-api-access-2gxww\") pod \"downloads-685bfb5f64-hkrq2\" (UID: \"2afbac7d-2575-439e-b7a5-f66063a6417a\") " pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:29.229839 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.229381 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gxww\" (UniqueName: \"kubernetes.io/projected/2afbac7d-2575-439e-b7a5-f66063a6417a-kube-api-access-2gxww\") pod \"downloads-685bfb5f64-hkrq2\" (UID: \"2afbac7d-2575-439e-b7a5-f66063a6417a\") " pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:29.243375 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.243264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gxww\" (UniqueName: \"kubernetes.io/projected/2afbac7d-2575-439e-b7a5-f66063a6417a-kube-api-access-2gxww\") pod \"downloads-685bfb5f64-hkrq2\" (UID: \"2afbac7d-2575-439e-b7a5-f66063a6417a\") " pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:29.396913 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.396824 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:29.571045 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:29.570966 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-hkrq2"] Jul 05 11:54:30.101049 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:30.101018 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2afbac7d_2575_439e_b7a5_f66063a6417a.slice/crio-77cca4f241057885cfa7555342d7582968bbf52b9a9b5f3319489d7464e30ab2 WatchSource:0}: Error finding container 77cca4f241057885cfa7555342d7582968bbf52b9a9b5f3319489d7464e30ab2: Status 404 returned error can't find the container with id 77cca4f241057885cfa7555342d7582968bbf52b9a9b5f3319489d7464e30ab2 Jul 05 11:54:30.617936 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:30.617893 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-hkrq2" event={"ID":"2afbac7d-2575-439e-b7a5-f66063a6417a","Type":"ContainerStarted","Data":"77cca4f241057885cfa7555342d7582968bbf52b9a9b5f3319489d7464e30ab2"} Jul 05 11:54:33.629412 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.629049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xs6k6" event={"ID":"567bbb08-f82e-4ce3-bf69-2c3e236990e2","Type":"ContainerStarted","Data":"27ac68c5c0a7047dfca84418dc06abc16f3a41e16bd052542dd266447557f78d"} Jul 05 11:54:33.631315 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.631287 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cl6j9" event={"ID":"0e06f685-a431-4af2-adfe-d13f498b1d26","Type":"ContainerStarted","Data":"3b0ea5cb0762dbfd3a6a2ccfdc9a7898446de8b61e41e29ebbf40417f130706c"} Jul 05 11:54:33.633292 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.633255 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n87rx" event={"ID":"9a8e5964-b285-4b87-b31d-6a165e12c20b","Type":"ContainerStarted","Data":"68fbe2cd7c7da51dd8cfc03d7b18fe7158893c00dd5a2e1cb1ba3266055c505a"} Jul 05 11:54:33.633416 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.633308 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n87rx" event={"ID":"9a8e5964-b285-4b87-b31d-6a165e12c20b","Type":"ContainerStarted","Data":"692834a4d026877f810b7502593a78072ca8c2b5ecfa97bce2d724ea8bc1ae09"} Jul 05 11:54:33.633474 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.633461 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:33.643489 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.643433 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xs6k6" podStartSLOduration=1.063408477 podStartE2EDuration="6.643417365s" totalImagesPullingTime="5.580008888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:33.642649515 +0000 UTC m=+57.900859975" watchObservedRunningTime="2026-07-05 11:54:33.643417365 +0000 UTC m=+57.901627823" Jul 05 11:54:33.658735 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:33.658687 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n87rx" podStartSLOduration=1.227230273 podStartE2EDuration="6.658669421s" totalImagesPullingTime="5.431439148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:33.657157166 +0000 UTC m=+57.915367626" watchObservedRunningTime="2026-07-05 11:54:33.658669421 +0000 UTC m=+57.916879882" Jul 05 11:54:34.640041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:34.640000 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cl6j9" event={"ID":"0e06f685-a431-4af2-adfe-d13f498b1d26","Type":"ContainerStarted","Data":"3b360139c7c82218d7d9bd08c032bea9ff86848090b6744d5fe9357d1a699892"} Jul 05 11:54:37.554293 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:37.554261 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djdvv" Jul 05 11:54:37.654398 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:37.654359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cl6j9" event={"ID":"0e06f685-a431-4af2-adfe-d13f498b1d26","Type":"ContainerStarted","Data":"602313a3b029768646a273f416461877796d9f6950993b6528ee101d0c86f395"} Jul 05 11:54:37.672347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:37.672291 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cl6j9" podStartSLOduration=1.672325014 podStartE2EDuration="10.672269638s" totalImagesPullingTime="8.999944624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:54:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:37.671198673 +0000 UTC m=+61.929409157" watchObservedRunningTime="2026-07-05 11:54:37.672269638 +0000 UTC m=+61.930480096" Jul 05 11:54:41.935370 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:41.935327 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:41.938174 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:41.938143 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 05 11:54:41.948822 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:41.948788 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f110573-9832-4310-93b6-1bc140a48214-metrics-certs\") pod \"network-metrics-daemon-29x85\" (UID: \"6f110573-9832-4310-93b6-1bc140a48214\") " pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:42.036708 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.036661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:42.036904 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.036725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:42.039697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.039662 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 05 11:54:42.039856 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.039704 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 05 11:54:42.049439 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.049411 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 05 11:54:42.049677 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.049654 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73b426e4-95af-454b-a6d5-50e35d34a4fa-original-pull-secret\") pod \"global-pull-secret-syncer-prgqv\" (UID: \"73b426e4-95af-454b-a6d5-50e35d34a4fa\") " pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:42.054873 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.054849 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-prgqv" Jul 05 11:54:42.060454 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.060432 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7gwq\" (UniqueName: \"kubernetes.io/projected/1358a8e0-1566-42e0-9b2d-5de59627c67f-kube-api-access-c7gwq\") pod \"network-check-target-tsfb7\" (UID: \"1358a8e0-1566-42e0-9b2d-5de59627c67f\") " pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:42.064809 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.064775 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dsspw\"" Jul 05 11:54:42.071618 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.071598 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-759jc\"" Jul 05 11:54:42.072598 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.072574 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:42.078320 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:42.078291 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29x85" Jul 05 11:54:43.643215 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:43.643187 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n87rx" Jul 05 11:54:46.565158 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.565125 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:54:46.567842 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.567822 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.571782 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.571673 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tz7xq\"" Jul 05 11:54:46.571782 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.571723 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 11:54:46.571782 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.571778 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 11:54:46.572146 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.572123 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 11:54:46.572249 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.572211 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 11:54:46.572811 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.572792 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 11:54:46.576717 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.576700 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 11:54:46.578556 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.578532 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:54:46.668176 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668176 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxzjg\" (UniqueName: \"kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668375 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668205 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668375 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668408 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.668472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.668432 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.769927 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.769889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770101 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.769944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770212 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.770180 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770304 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.770276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770350 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.770312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxzjg\" (UniqueName: \"kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770401 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.770370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.770447 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.770401 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.771105 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.771040 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.771251 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.771139 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.771251 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.771197 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.771251 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.771235 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.772970 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.772948 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.773331 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.773312 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.777997 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.777944 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxzjg\" (UniqueName: \"kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg\") pod \"console-54bb4bddd8-7b966\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:46.880062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:46.879978 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:54:47.159472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.159387 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2"] Jul 05 11:54:47.165597 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.165566 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.168075 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168049 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 05 11:54:47.168206 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168113 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 05 11:54:47.168278 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168224 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 05 11:54:47.168278 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168266 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-l5zfp\"" Jul 05 11:54:47.168444 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168427 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 05 11:54:47.168529 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.168511 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 05 11:54:47.171088 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.170846 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2"] Jul 05 11:54:47.201355 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.201328 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-24zbx"] Jul 05 11:54:47.203448 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.203430 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.205834 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.205805 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 05 11:54:47.205925 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.205910 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 05 11:54:47.205994 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.205939 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mqzr7\"" Jul 05 11:54:47.206085 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.206055 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 05 11:54:47.274275 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.274275 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4c9j\" (UniqueName: \"kubernetes.io/projected/d495752d-bf11-4d9c-b0c4-3d0dd2937138-kube-api-access-w4c9j\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.274508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274304 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-root\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-metrics-client-ca\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d495752d-bf11-4d9c-b0c4-3d0dd2937138-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.274508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-textfile\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274458 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-wtmp\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274508 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274530 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-sys\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j6pb\" (UniqueName: \"kubernetes.io/projected/8a1de157-97c4-4256-bf25-9d837858e2af-kube-api-access-7j6pb\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.274698 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.274687 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-accelerators-collector-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376007 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.375955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.376007 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376004 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4c9j\" (UniqueName: \"kubernetes.io/projected/d495752d-bf11-4d9c-b0c4-3d0dd2937138-kube-api-access-w4c9j\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376031 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-root\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-metrics-client-ca\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376077 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d495752d-bf11-4d9c-b0c4-3d0dd2937138-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376104 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-textfile\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376129 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-wtmp\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376140 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-root\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376670 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376275 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-sys\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376670 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376305 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j6pb\" (UniqueName: \"kubernetes.io/projected/8a1de157-97c4-4256-bf25-9d837858e2af-kube-api-access-7j6pb\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.376670 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-accelerators-collector-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377000 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376980 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-textfile\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377000 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.376978 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-metrics-client-ca\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377128 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.377039 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-sys\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377128 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.377077 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-accelerators-collector-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377128 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.377102 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-wtmp\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.377128 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.377105 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d495752d-bf11-4d9c-b0c4-3d0dd2937138-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.377310 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:47.377243 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 05 11:54:47.377350 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:54:47.377312 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls podName:8a1de157-97c4-4256-bf25-9d837858e2af nodeName:}" failed. No retries permitted until 2026-07-05 11:54:47.877292809 +0000 UTC m=+72.135503248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls") pod "node-exporter-24zbx" (UID: "8a1de157-97c4-4256-bf25-9d837858e2af") : secret "node-exporter-tls" not found Jul 05 11:54:47.379093 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.379072 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.379213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.379148 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d495752d-bf11-4d9c-b0c4-3d0dd2937138-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.379614 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.379594 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.386898 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.386880 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4c9j\" (UniqueName: \"kubernetes.io/projected/d495752d-bf11-4d9c-b0c4-3d0dd2937138-kube-api-access-w4c9j\") pod \"openshift-state-metrics-54cf96dfcf-fk9z2\" (UID: \"d495752d-bf11-4d9c-b0c4-3d0dd2937138\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.387394 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.387375 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j6pb\" (UniqueName: \"kubernetes.io/projected/8a1de157-97c4-4256-bf25-9d837858e2af-kube-api-access-7j6pb\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.474332 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.474294 2574 patch_prober.go:28] interesting pod/image-registry-646c4846d9-bl7dd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 05 11:54:47.474497 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.474361 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" podUID="651e7d2b-fea9-419f-bd91-3b3ae586f620" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 05 11:54:47.477431 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.477406 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" Jul 05 11:54:47.881030 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.880935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:47.883482 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:47.883459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8a1de157-97c4-4256-bf25-9d837858e2af-node-exporter-tls\") pod \"node-exporter-24zbx\" (UID: \"8a1de157-97c4-4256-bf25-9d837858e2af\") " pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:48.113031 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:48.112996 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-24zbx" Jul 05 11:54:48.585946 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:48.585621 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1de157_97c4_4256_bf25_9d837858e2af.slice/crio-c6aaf4db44e506f8ecd37ddf076ac703f93f6536256f9ac4d94020c701e2679e WatchSource:0}: Error finding container c6aaf4db44e506f8ecd37ddf076ac703f93f6536256f9ac4d94020c701e2679e: Status 404 returned error can't find the container with id c6aaf4db44e506f8ecd37ddf076ac703f93f6536256f9ac4d94020c701e2679e Jul 05 11:54:48.702793 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:48.702697 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24zbx" event={"ID":"8a1de157-97c4-4256-bf25-9d837858e2af","Type":"ContainerStarted","Data":"c6aaf4db44e506f8ecd37ddf076ac703f93f6536256f9ac4d94020c701e2679e"} Jul 05 11:54:48.746385 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:48.746294 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2"] Jul 05 11:54:48.751373 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:48.751337 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd495752d_bf11_4d9c_b0c4_3d0dd2937138.slice/crio-d38c249f2a2b43f7581411b4e7d78f8bd1265c32b1ff9b0c8458a944702820bb WatchSource:0}: Error finding container d38c249f2a2b43f7581411b4e7d78f8bd1265c32b1ff9b0c8458a944702820bb: Status 404 returned error can't find the container with id d38c249f2a2b43f7581411b4e7d78f8bd1265c32b1ff9b0c8458a944702820bb Jul 05 11:54:48.767812 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:48.767693 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsfb7"] Jul 05 11:54:48.772269 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:48.772240 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1358a8e0_1566_42e0_9b2d_5de59627c67f.slice/crio-ce6985a942ef7611ef63cd62c9e03791a22605aecb0cfe878bdd7f981d617c11 WatchSource:0}: Error finding container ce6985a942ef7611ef63cd62c9e03791a22605aecb0cfe878bdd7f981d617c11: Status 404 returned error can't find the container with id ce6985a942ef7611ef63cd62c9e03791a22605aecb0cfe878bdd7f981d617c11 Jul 05 11:54:49.001587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.001433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-prgqv"] Jul 05 11:54:49.004636 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.004589 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29x85"] Jul 05 11:54:49.005586 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.005561 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:54:49.012499 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:49.012461 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f110573_9832_4310_93b6_1bc140a48214.slice/crio-c8d3e2e4dffcd28e32bad8e4cbc32212e29e4c2bc0c76c61980c35d5e1d9abce WatchSource:0}: Error finding container c8d3e2e4dffcd28e32bad8e4cbc32212e29e4c2bc0c76c61980c35d5e1d9abce: Status 404 returned error can't find the container with id c8d3e2e4dffcd28e32bad8e4cbc32212e29e4c2bc0c76c61980c35d5e1d9abce Jul 05 11:54:49.013800 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:49.013778 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30768041_23e0_444f_873e_1d8c0169dd1e.slice/crio-ac1390e9ac0a5267fbb8bef8802d29e3ad8cd5f9aa427865dab5c0838436165f WatchSource:0}: Error finding container ac1390e9ac0a5267fbb8bef8802d29e3ad8cd5f9aa427865dab5c0838436165f: Status 404 returned error can't find the container with id ac1390e9ac0a5267fbb8bef8802d29e3ad8cd5f9aa427865dab5c0838436165f Jul 05 11:54:49.621792 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.621556 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-646c4846d9-bl7dd" Jul 05 11:54:49.718719 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.718676 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-prgqv" event={"ID":"73b426e4-95af-454b-a6d5-50e35d34a4fa","Type":"ContainerStarted","Data":"64739ee3eb61c9969bdb994ae75d23f3b13775515ce63112edd91b21647be4d4"} Jul 05 11:54:49.720993 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.720961 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29x85" event={"ID":"6f110573-9832-4310-93b6-1bc140a48214","Type":"ContainerStarted","Data":"c8d3e2e4dffcd28e32bad8e4cbc32212e29e4c2bc0c76c61980c35d5e1d9abce"} Jul 05 11:54:49.723263 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.723236 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-hkrq2" event={"ID":"2afbac7d-2575-439e-b7a5-f66063a6417a","Type":"ContainerStarted","Data":"547358ebdaf0abafede2b2926528aa70939aab44ee1ed5ce1a8247bc1abf6bed"} Jul 05 11:54:49.724139 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.724118 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:49.726414 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.726384 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bb4bddd8-7b966" event={"ID":"30768041-23e0-444f-873e-1d8c0169dd1e","Type":"ContainerStarted","Data":"ac1390e9ac0a5267fbb8bef8802d29e3ad8cd5f9aa427865dab5c0838436165f"} Jul 05 11:54:49.728664 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.728638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsfb7" event={"ID":"1358a8e0-1566-42e0-9b2d-5de59627c67f","Type":"ContainerStarted","Data":"ce6985a942ef7611ef63cd62c9e03791a22605aecb0cfe878bdd7f981d617c11"} Jul 05 11:54:49.732512 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.732485 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" event={"ID":"d495752d-bf11-4d9c-b0c4-3d0dd2937138","Type":"ContainerStarted","Data":"097d7a94e31c476f33b3d101d6227efc26d1a30b4de979cbd21ee3b402388a4a"} Jul 05 11:54:49.732609 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.732545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" event={"ID":"d495752d-bf11-4d9c-b0c4-3d0dd2937138","Type":"ContainerStarted","Data":"0589691b1401657a6accc81d4fd4ef2e5dc152f25ff40eba96e413bdd403fe61"} Jul 05 11:54:49.732609 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.732579 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" event={"ID":"d495752d-bf11-4d9c-b0c4-3d0dd2937138","Type":"ContainerStarted","Data":"d38c249f2a2b43f7581411b4e7d78f8bd1265c32b1ff9b0c8458a944702820bb"} Jul 05 11:54:49.739591 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.739543 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-hkrq2" Jul 05 11:54:49.760839 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:49.759906 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-hkrq2" podStartSLOduration=2.195925032 podStartE2EDuration="20.759885254s" totalImagesPullingTime="18.563960222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:49.74186714 +0000 UTC m=+74.000077666" watchObservedRunningTime="2026-07-05 11:54:49.759885254 +0000 UTC m=+74.018095716" Jul 05 11:54:50.163041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.161339 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5b9698864d-5z6b7"] Jul 05 11:54:50.176798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.175369 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.176798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.176537 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9698864d-5z6b7"] Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.178570 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jmgjc\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.178887 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.179079 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.179288 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.179481 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.179808 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-90kcbhg3v8i8s\"" Jul 05 11:54:50.180141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.179995 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psfl9\" (UniqueName: \"kubernetes.io/projected/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-kube-api-access-psfl9\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308349 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308388 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308413 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308448 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-grpc-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308478 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-metrics-client-ca\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.308697 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.308576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409684 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psfl9\" (UniqueName: \"kubernetes.io/projected/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-kube-api-access-psfl9\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409803 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409862 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-grpc-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.411166 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.409960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-metrics-client-ca\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.413319 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.413203 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-metrics-client-ca\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.420179 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.420127 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.422994 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.422550 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.423839 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.423521 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.423839 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.423630 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.424391 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.424341 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-thanos-querier-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.433073 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.432839 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-secret-grpc-tls\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.450726 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.450657 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psfl9\" (UniqueName: \"kubernetes.io/projected/a6262d58-6df7-4fac-b93e-54e6a7fe3cbf-kube-api-access-psfl9\") pod \"thanos-querier-5b9698864d-5z6b7\" (UID: \"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf\") " pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.496931 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.496630 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:54:50.735496 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.735461 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5b9698864d-5z6b7"] Jul 05 11:54:50.747583 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:54:50.747553 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6262d58_6df7_4fac_b93e_54e6a7fe3cbf.slice/crio-f2e74c546d266ba8c2f7d7461ab28dcf0d980ab493f118df4c2344c63cd01240 WatchSource:0}: Error finding container f2e74c546d266ba8c2f7d7461ab28dcf0d980ab493f118df4c2344c63cd01240: Status 404 returned error can't find the container with id f2e74c546d266ba8c2f7d7461ab28dcf0d980ab493f118df4c2344c63cd01240 Jul 05 11:54:50.751729 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.751697 2574 generic.go:356] "Generic (PLEG): container finished" podID="8a1de157-97c4-4256-bf25-9d837858e2af" containerID="2936cb0f817256c23c6560e8e239db148517b728bc5707ec9e4f29d3b1a48ab4" exitCode=0 Jul 05 11:54:50.752818 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:50.752734 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24zbx" event={"ID":"8a1de157-97c4-4256-bf25-9d837858e2af","Type":"ContainerDied","Data":"2936cb0f817256c23c6560e8e239db148517b728bc5707ec9e4f29d3b1a48ab4"} Jul 05 11:54:51.762202 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:51.762053 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"f2e74c546d266ba8c2f7d7461ab28dcf0d980ab493f118df4c2344c63cd01240"} Jul 05 11:54:51.770540 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:51.770448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24zbx" event={"ID":"8a1de157-97c4-4256-bf25-9d837858e2af","Type":"ContainerStarted","Data":"cd76c2950df4d76d4080ad52dbb30425220dd95fd0857f57aac1d6a4ba0f3904"} Jul 05 11:54:52.976149 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:52.976022 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:54:53.534261 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.534224 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:54:53.563833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.562505 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:54:53.563833 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.562687 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.567041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.566133 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 11:54:53.567041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.566249 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 11:54:53.567041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.566427 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 11:54:53.567041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.566657 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 11:54:53.567041 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.566716 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 11:54:53.567346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.567187 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 11:54:53.567821 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.567405 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 11:54:53.567821 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.567703 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 11:54:53.568590 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.568193 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 11:54:53.568590 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.568431 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 11:54:53.568590 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.568437 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 11:54:53.569134 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.568602 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 11:54:53.569654 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.569370 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rf046gjf6ve6\"" Jul 05 11:54:53.569654 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.569438 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rsl6v\"" Jul 05 11:54:53.571941 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.571691 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 11:54:53.576816 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.576678 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 11:54:53.578994 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.578974 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 11:54:53.645405 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645348 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645405 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645431 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645483 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645506 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.645650 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvwp\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646007 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.645969 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646093 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646071 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646158 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646229 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646193 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646284 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646243 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646284 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646384 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646384 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646342 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646384 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646366 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646525 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646392 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646525 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646422 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.646525 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.646450 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.747831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747788 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747838 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747925 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747967 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.748015 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.747993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvwp\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.748791 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749431 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749474 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749513 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749580 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749646 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749709 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749777 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749805 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.751472 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.749858 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.758213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.750860 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.758213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.755006 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.758213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.756248 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.758213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.756779 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.758213 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.757245 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.760798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.758996 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvwp\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.760798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.759688 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.760798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.760217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.760798 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.760484 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.762072 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.762048 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.763558 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.763517 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.767070 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.767007 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.771051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.770324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.771051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.770522 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.771051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.770755 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.771051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.771001 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.772229 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.772187 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.773926 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.773896 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.778680 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.778650 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:53.877700 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:53.877616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:54:58.965795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:58.963004 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:54:59.814940 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.814702 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" event={"ID":"d495752d-bf11-4d9c-b0c4-3d0dd2937138","Type":"ContainerStarted","Data":"bcf3fcb4d5b17842ef41fd342e4883fe8d1b02b0614de6ce56d9ff1134b692f2"} Jul 05 11:54:59.816417 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.816388 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"c6911ea87520b0d93ea9440430fdaab435b6f5d7e1c09fee66010f0e93753134"} Jul 05 11:54:59.819170 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.819142 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"53b7c0bd322fcee0682e1d5fb1a322834782220936d07a5bfb80c399db160887"} Jul 05 11:54:59.819297 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.819216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"fca52a0b20754fd0666d453e9690ad8858c5123fe6885267c38d3ae0bc07882c"} Jul 05 11:54:59.819297 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.819260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"ac6a05446f863494028bb20466db2d85c45ee7fd86760072091f1c4d69abc6f9"} Jul 05 11:54:59.821920 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.821892 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-24zbx" event={"ID":"8a1de157-97c4-4256-bf25-9d837858e2af","Type":"ContainerStarted","Data":"f8ef9381f74e5810f8d4b9a2f76a1f634b8da4e9a65dcf91e8b74fdf929ac7ec"} Jul 05 11:54:59.823690 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.823620 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-prgqv" event={"ID":"73b426e4-95af-454b-a6d5-50e35d34a4fa","Type":"ContainerStarted","Data":"9836df059333169c1174d733d51953c92335ac4401adb6248bddb1944ce5d6f8"} Jul 05 11:54:59.825922 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.825899 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29x85" event={"ID":"6f110573-9832-4310-93b6-1bc140a48214","Type":"ContainerStarted","Data":"9e681fcd06d1bc7fa2ffd9fc5bccb5185d7a572a82c0fb1bc65301e679750133"} Jul 05 11:54:59.826054 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.825939 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29x85" event={"ID":"6f110573-9832-4310-93b6-1bc140a48214","Type":"ContainerStarted","Data":"0654da4648daf3299737b0636250849d3cbcbaf7bd3d76956e73e59c7e849712"} Jul 05 11:54:59.828123 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.828091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bb4bddd8-7b966" event={"ID":"30768041-23e0-444f-873e-1d8c0169dd1e","Type":"ContainerStarted","Data":"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a"} Jul 05 11:54:59.829729 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.829706 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsfb7" event={"ID":"1358a8e0-1566-42e0-9b2d-5de59627c67f","Type":"ContainerStarted","Data":"d3fbddb231f3286f6f29d33a2a04fac62776eaad61c95997bbccbe9cdb65b0e8"} Jul 05 11:54:59.829844 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.829808 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:54:59.852154 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.852104 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-prgqv" podStartSLOduration=74.06946661 podStartE2EDuration="1m23.852085082s" totalImagesPullingTime="9.782618472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.851673224 +0000 UTC m=+84.109883711" watchObservedRunningTime="2026-07-05 11:54:59.852085082 +0000 UTC m=+84.110295566" Jul 05 11:54:59.852528 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.852496 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fk9z2" podStartSLOduration=2.985701169 podStartE2EDuration="12.852487542s" totalImagesPullingTime="9.866786373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.833337228 +0000 UTC m=+84.091547712" watchObservedRunningTime="2026-07-05 11:54:59.852487542 +0000 UTC m=+84.110698001" Jul 05 11:54:59.870226 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.870175 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-24zbx" podStartSLOduration=11.414007935 podStartE2EDuration="12.870159707s" totalImagesPullingTime="1.323095371s" totalInitContainerRuntime="133.056401ms" isStatefulPod=true podCreationTimestamp="2026-07-05 11:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.869029219 +0000 UTC m=+84.127239707" watchObservedRunningTime="2026-07-05 11:54:59.870159707 +0000 UTC m=+84.128370166" Jul 05 11:54:59.886508 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.886463 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54bb4bddd8-7b966" podStartSLOduration=4.09917187 podStartE2EDuration="13.886448161s" totalImagesPullingTime="9.787276291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.885545352 +0000 UTC m=+84.143755844" watchObservedRunningTime="2026-07-05 11:54:59.886448161 +0000 UTC m=+84.144658621" Jul 05 11:54:59.902347 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.902292 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29x85" podStartSLOduration=75.751697985 podStartE2EDuration="1m23.902278065s" totalImagesPullingTime="8.15058008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.901916367 +0000 UTC m=+84.160126868" watchObservedRunningTime="2026-07-05 11:54:59.902278065 +0000 UTC m=+84.160488525" Jul 05 11:54:59.917429 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:54:59.917381 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tsfb7" podStartSLOduration=73.89770135 podStartE2EDuration="1m23.917367178s" totalImagesPullingTime="10.019665828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:54:59.916209531 +0000 UTC m=+84.174420018" watchObservedRunningTime="2026-07-05 11:54:59.917367178 +0000 UTC m=+84.175577638" Jul 05 11:55:00.834078 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:00.834047 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" exitCode=0 Jul 05 11:55:00.834396 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:00.834132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} Jul 05 11:55:00.836622 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:00.836595 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"208ace4ad9f74be1d69ff6f160cf7ba5166ed2a91b83bf9733f03c4c854d7cfa"} Jul 05 11:55:01.844368 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:01.844326 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"4805512b47e5c379e5c8a8ea11e91fba492103021fea36afb9d5f52ec0c7d96f"} Jul 05 11:55:01.844829 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:01.844448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" event={"ID":"a6262d58-6df7-4fac-b93e-54e6a7fe3cbf","Type":"ContainerStarted","Data":"dc47f8c6b6a84f00213361a05d19634ce6e259a03dccaf8e0cfffa887af8888e"} Jul 05 11:55:01.844829 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:01.844539 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:55:01.866848 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:01.866784 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" podStartSLOduration=2.427309706 podStartE2EDuration="11.866745575s" totalImagesPullingTime="9.439435869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:55:01.86494858 +0000 UTC m=+86.123159062" watchObservedRunningTime="2026-07-05 11:55:01.866745575 +0000 UTC m=+86.124956034" Jul 05 11:55:04.859447 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:04.859409 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} Jul 05 11:55:04.859784 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:04.859500 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} Jul 05 11:55:04.859784 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:04.859525 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} Jul 05 11:55:04.859784 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:04.859551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} Jul 05 11:55:04.859784 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:04.859574 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} Jul 05 11:55:05.865772 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:05.865721 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerStarted","Data":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} Jul 05 11:55:05.894866 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:05.894818 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.773932167 podStartE2EDuration="12.894800418s" totalImagesPullingTime="5.105681509s" totalInitContainerRuntime="15.186742ms" isStatefulPod=false podCreationTimestamp="2026-07-05 11:54:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:55:05.892181293 +0000 UTC m=+90.150391769" watchObservedRunningTime="2026-07-05 11:55:05.894800418 +0000 UTC m=+90.153010877" Jul 05 11:55:06.880413 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:06.880380 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:55:07.855618 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:07.855593 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5b9698864d-5z6b7" Jul 05 11:55:08.878081 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:08.878049 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:55:24.851454 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:24.851390 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-54bb4bddd8-7b966" podUID="30768041-23e0-444f-873e-1d8c0169dd1e" containerName="console" containerID="cri-o://33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a" gracePeriod=15 Jul 05 11:55:25.086696 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.086669 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:55:25.118096 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118032 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118096 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118084 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118244 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118131 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118244 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118168 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118244 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118211 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118244 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118242 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxzjg\" (UniqueName: \"kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118448 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118313 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config\") pod \"30768041-23e0-444f-873e-1d8c0169dd1e\" (UID: \"30768041-23e0-444f-873e-1d8c0169dd1e\") " Jul 05 11:55:25.118582 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118534 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:25.118797 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118652 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca" (OuterVolumeSpecName: "service-ca") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:25.118797 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118755 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config" (OuterVolumeSpecName: "console-config") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:25.118921 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.118872 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:55:25.120402 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.120377 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:55:25.120502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.120443 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg" (OuterVolumeSpecName: "kube-api-access-fxzjg") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "kube-api-access-fxzjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:55:25.120815 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.120797 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "30768041-23e0-444f-873e-1d8c0169dd1e" (UID: "30768041-23e0-444f-873e-1d8c0169dd1e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:55:25.219846 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219821 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-oauth-serving-cert\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219846 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219844 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-console-config\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219853 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxzjg\" (UniqueName: \"kubernetes.io/projected/30768041-23e0-444f-873e-1d8c0169dd1e-kube-api-access-fxzjg\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219863 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-oauth-config\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219873 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-trusted-ca-bundle\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219881 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30768041-23e0-444f-873e-1d8c0169dd1e-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.219984 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.219889 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30768041-23e0-444f-873e-1d8c0169dd1e-console-serving-cert\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:55:25.941470 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.941436 2574 generic.go:356] "Generic (PLEG): container finished" podID="30768041-23e0-444f-873e-1d8c0169dd1e" containerID="33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a" exitCode=0 Jul 05 11:55:25.941957 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.941483 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bb4bddd8-7b966" Jul 05 11:55:25.941957 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.941528 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bb4bddd8-7b966" event={"ID":"30768041-23e0-444f-873e-1d8c0169dd1e","Type":"ContainerDied","Data":"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a"} Jul 05 11:55:25.941957 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.941585 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bb4bddd8-7b966" event={"ID":"30768041-23e0-444f-873e-1d8c0169dd1e","Type":"ContainerDied","Data":"ac1390e9ac0a5267fbb8bef8802d29e3ad8cd5f9aa427865dab5c0838436165f"} Jul 05 11:55:25.941957 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.941615 2574 scope.go:117] "RemoveContainer" containerID="33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a" Jul 05 11:55:25.950172 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.950155 2574 scope.go:117] "RemoveContainer" containerID="33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a" Jul 05 11:55:25.950423 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:55:25.950404 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a\": container with ID starting with 33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a not found: ID does not exist" containerID="33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a" Jul 05 11:55:25.950506 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.950431 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a"} err="failed to get container status \"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a\": rpc error: code = NotFound desc = could not find container \"33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a\": container with ID starting with 33f23636838b1760e8fd5b6f0102bda3e60ffa7c0de4d9ffe2697ba28785cd8a not found: ID does not exist" Jul 05 11:55:25.961615 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.961594 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:55:25.972277 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:25.972252 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54bb4bddd8-7b966"] Jul 05 11:55:26.242369 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:26.242289 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30768041-23e0-444f-873e-1d8c0169dd1e" path="/var/lib/kubelet/pods/30768041-23e0-444f-873e-1d8c0169dd1e/volumes" Jul 05 11:55:30.839968 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:30.839941 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tsfb7" Jul 05 11:55:53.878149 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:53.878097 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:55:53.897059 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:53.897028 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:55:54.057211 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:55:54.057186 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:11.722947 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.722914 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723256 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" gracePeriod=600 Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723269 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="thanos-sidecar" containerID="cri-o://d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" gracePeriod=600 Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723302 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-web" containerID="cri-o://f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" gracePeriod=600 Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723245 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy" containerID="cri-o://e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" gracePeriod=600 Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723288 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="config-reloader" containerID="cri-o://03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" gracePeriod=600 Jul 05 11:56:11.723657 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.723239 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="prometheus" containerID="cri-o://50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" gracePeriod=600 Jul 05 11:56:11.957154 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:11.957127 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.090799 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090706 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.090799 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090744 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.090799 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090792 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090900 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090951 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.090997 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091023 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091062 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091057 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091300 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091093 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091300 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091128 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091300 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091146 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.091300 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091156 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091312 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091349 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091382 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091409 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091439 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfvwp\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091469 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091502 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091499 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091802 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091545 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.091802 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.091570 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0\") pod \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\" (UID: \"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf\") " Jul 05 11:56:12.093208 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.092219 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.093338 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.093267 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.093413 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.093296 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 11:56:12.093473 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.093453 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.093868 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.093839 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.095485 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.095446 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.096021 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.095922 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.096127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096082 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.096191 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096131 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.096191 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096159 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out" (OuterVolumeSpecName: "config-out") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 11:56:12.096390 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096321 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.096539 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096516 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 11:56:12.096662 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096637 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.096893 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.096871 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097021 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097009 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config-out\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097121 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097109 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097186 2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097289 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097270 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097290 2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-grpc-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097309 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097322 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-metrics-client-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097336 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097349 2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-metrics-client-certs\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097378 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097368 2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-prometheus-k8s-db\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097655 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097382 2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097655 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097397 2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-kube-rbac-proxy\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097655 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097411 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.097655 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097477 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp" (OuterVolumeSpecName: "kube-api-access-mfvwp") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "kube-api-access-mfvwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:56:12.098025 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.097997 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.098225 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.098201 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.099923 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.099897 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 11:56:12.101448 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.101428 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config" (OuterVolumeSpecName: "config") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.108099 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.108073 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config" (OuterVolumeSpecName: "web-config") pod "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" (UID: "ae5e2529-5022-4bb2-a3ea-231bf85d3eaf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 11:56:12.112034 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112011 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" exitCode=0 Jul 05 11:56:12.112034 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112032 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" exitCode=0 Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112039 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" exitCode=0 Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112044 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" exitCode=0 Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112049 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" exitCode=0 Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112054 2574 generic.go:356] "Generic (PLEG): container finished" podID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" exitCode=0 Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112072 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} Jul 05 11:56:12.112143 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112092 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112177 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112219 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112248 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} Jul 05 11:56:12.112321 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.112271 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae5e2529-5022-4bb2-a3ea-231bf85d3eaf","Type":"ContainerDied","Data":"c6911ea87520b0d93ea9440430fdaab435b6f5d7e1c09fee66010f0e93753134"} Jul 05 11:56:12.120632 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.120610 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.127351 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.127335 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.133451 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.133436 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.136007 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.135987 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:12.139635 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.139615 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:12.140831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.140818 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.147048 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.147032 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.153655 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.153638 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.161661 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.161636 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.161986 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.161958 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.162066 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.161999 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.162066 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162024 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.162334 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.162307 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.162436 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162338 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.162436 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162355 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.162641 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.162621 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.162681 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162647 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.162681 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162665 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.162946 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.162929 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.163004 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162952 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.163004 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.162968 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.163188 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163172 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:12.163233 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.163209 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.163267 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163230 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.163267 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163246 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.163443 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163428 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="prometheus" Jul 05 11:56:12.163480 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163446 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="prometheus" Jul 05 11:56:12.163480 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163462 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="thanos-sidecar" Jul 05 11:56:12.163480 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163471 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="thanos-sidecar" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163481 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="init-config-reloader" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.163477 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163487 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="init-config-reloader" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163494 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-web" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163499 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-web" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163495 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163506 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163509 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163511 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163520 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30768041-23e0-444f-873e-1d8c0169dd1e" containerName="console" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163525 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="30768041-23e0-444f-873e-1d8c0169dd1e" containerName="console" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163532 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="config-reloader" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163540 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="config-reloader" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163547 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-thanos" Jul 05 11:56:12.163563 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163553 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-thanos" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163613 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="30768041-23e0-444f-873e-1d8c0169dd1e" containerName="console" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163625 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="prometheus" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163634 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="thanos-sidecar" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163643 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="config-reloader" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163652 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-web" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163660 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163668 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" containerName="kube-rbac-proxy-thanos" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: E0705 11:56:12.163753 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163785 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.163798 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164014 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.164127 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164029 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.164520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164226 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.164520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164240 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.164520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164420 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.164520 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164437 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.164654 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164610 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.164654 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164623 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.164889 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164870 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.164936 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.164889 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.165122 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165103 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.165196 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165124 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.165391 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165367 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.165391 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165387 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.165626 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165606 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.165679 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165627 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.165858 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165842 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.165858 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.165858 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.166083 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166065 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.166134 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166084 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.166305 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166286 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.166346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166304 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.166510 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166494 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.166568 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166511 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.166719 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166704 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.166775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166719 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.166956 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166940 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.167008 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.166956 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.167190 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167167 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.167240 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167191 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.167355 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167340 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.167406 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167371 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.167406 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167385 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.167732 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167589 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.167732 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167621 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.168009 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.167987 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.168063 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168012 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.168285 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168263 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.168285 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168285 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.168494 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168476 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.168554 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168494 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.168724 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168699 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.168724 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168720 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.168981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168961 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.169032 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.168982 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.169214 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169197 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.169284 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169217 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.169452 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169437 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.169516 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169454 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.169685 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169662 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.169741 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169689 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.169976 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169956 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.170049 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.169976 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.170049 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170016 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 05 11:56:12.170141 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170077 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 05 11:56:12.170215 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170162 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 05 11:56:12.170303 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170269 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 05 11:56:12.170303 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170271 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 05 11:56:12.170404 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170323 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 05 11:56:12.170404 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170317 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.170404 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170370 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.170404 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170379 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 05 11:56:12.170775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170676 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 05 11:56:12.170775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170713 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 05 11:56:12.170775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170739 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rsl6v\"" Jul 05 11:56:12.170775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170739 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 05 11:56:12.171001 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170714 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 05 11:56:12.171001 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170739 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 05 11:56:12.171096 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.170996 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.171096 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171023 2574 scope.go:117] "RemoveContainer" containerID="ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792" Jul 05 11:56:12.171202 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171125 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 05 11:56:12.171256 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171240 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4rf046gjf6ve6\"" Jul 05 11:56:12.171306 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171286 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792"} err="failed to get container status \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": rpc error: code = NotFound desc = could not find container \"ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792\": container with ID starting with ee8a31a2b61f51432d3d449a103b10ecd3d4adaa584f9aae09dc7f9410f85792 not found: ID does not exist" Jul 05 11:56:12.171348 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171308 2574 scope.go:117] "RemoveContainer" containerID="e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71" Jul 05 11:56:12.171531 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171510 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71"} err="failed to get container status \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": rpc error: code = NotFound desc = could not find container \"e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71\": container with ID starting with e99c8383c8ff8969dc6f8d500793ee1ada955d131ad583093ac97b851beb3b71 not found: ID does not exist" Jul 05 11:56:12.171531 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171531 2574 scope.go:117] "RemoveContainer" containerID="f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d" Jul 05 11:56:12.171797 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171752 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d"} err="failed to get container status \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": rpc error: code = NotFound desc = could not find container \"f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d\": container with ID starting with f25090da0a61cb07f72e0fd0eac0af22e256e0c37b8b1e03cf45da957323470d not found: ID does not exist" Jul 05 11:56:12.171877 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.171798 2574 scope.go:117] "RemoveContainer" containerID="d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510" Jul 05 11:56:12.172056 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172029 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510"} err="failed to get container status \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": rpc error: code = NotFound desc = could not find container \"d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510\": container with ID starting with d4ab49dcb7eb4cfece734678c9829a3480834861e24d20c6e715e9dbad0d6510 not found: ID does not exist" Jul 05 11:56:12.172160 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172057 2574 scope.go:117] "RemoveContainer" containerID="03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f" Jul 05 11:56:12.172283 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172260 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f"} err="failed to get container status \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": rpc error: code = NotFound desc = could not find container \"03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f\": container with ID starting with 03c9db0004a0920e71da9ba4184e812365e1e85cdc13b4ad95d246f9d8146f5f not found: ID does not exist" Jul 05 11:56:12.172356 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172286 2574 scope.go:117] "RemoveContainer" containerID="50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0" Jul 05 11:56:12.172640 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172616 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0"} err="failed to get container status \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": rpc error: code = NotFound desc = could not find container \"50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0\": container with ID starting with 50b78cd338d74d52723a9aac756948fe5f9f64ca34531b586a516c8182f7c7f0 not found: ID does not exist" Jul 05 11:56:12.172702 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172642 2574 scope.go:117] "RemoveContainer" containerID="acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb" Jul 05 11:56:12.172974 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.172931 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb"} err="failed to get container status \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": rpc error: code = NotFound desc = could not find container \"acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb\": container with ID starting with acbe763a9621f870eaca6055d9124f04b5fbdffc00cf3ca2821d19f90b0d1dfb not found: ID does not exist" Jul 05 11:56:12.173938 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.173910 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 05 11:56:12.175595 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.175577 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 05 11:56:12.185400 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.185381 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:12.198257 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198237 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bphst\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-kube-api-access-bphst\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198262 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198278 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-web-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198346 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198475 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198403 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198475 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198475 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198475 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198516 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198538 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198565 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config-out\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198587 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198583 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198707 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198795 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198918 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-web-config\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198935 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-tls-assets\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198949 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198964 2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-config\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198978 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfvwp\" (UniqueName: \"kubernetes.io/projected/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-kube-api-access-mfvwp\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.198996 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.198991 2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 11:56:12.243192 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.243164 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae5e2529-5022-4bb2-a3ea-231bf85d3eaf" path="/var/lib/kubelet/pods/ae5e2529-5022-4bb2-a3ea-231bf85d3eaf/volumes" Jul 05 11:56:12.299959 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.299934 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config-out\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300026 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.299963 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300026 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.299983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300026 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300001 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300052 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300084 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300114 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300136 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300163 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bphst\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-kube-api-access-bphst\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300178 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300181 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300196 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-web-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300232 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300292 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300432 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.300492 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301053 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.300731 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301273 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.301247 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301336 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.301309 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301409 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.301388 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301870 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.301844 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.301981 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.301933 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.302907 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.302879 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config-out\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.303509 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.303453 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.304455 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304423 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.304550 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304530 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.304744 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304721 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.304831 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304742 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305001 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304980 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-web-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305001 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.304990 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305103 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.305056 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305335 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.305319 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-config\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305706 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.305683 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.305987 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.305972 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.309078 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.309060 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bphst\" (UniqueName: \"kubernetes.io/projected/5db51df0-29ab-4ecb-b550-ebca09ba4eb6-kube-api-access-bphst\") pod \"prometheus-k8s-0\" (UID: \"5db51df0-29ab-4ecb-b550-ebca09ba4eb6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.477958 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.477935 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:56:12.603883 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:12.603854 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 05 11:56:12.607026 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:56:12.606998 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db51df0_29ab_4ecb_b550_ebca09ba4eb6.slice/crio-78d27f21e3686c12a0a66c03637e9fc9b8b7a633c488d3efb8d3b7ae9bcb4382 WatchSource:0}: Error finding container 78d27f21e3686c12a0a66c03637e9fc9b8b7a633c488d3efb8d3b7ae9bcb4382: Status 404 returned error can't find the container with id 78d27f21e3686c12a0a66c03637e9fc9b8b7a633c488d3efb8d3b7ae9bcb4382 Jul 05 11:56:13.117052 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:13.116974 2574 generic.go:356] "Generic (PLEG): container finished" podID="5db51df0-29ab-4ecb-b550-ebca09ba4eb6" containerID="22eb87a6f6860ab39c4f72b6310338ee7936235405b8414189f74995fc750c16" exitCode=0 Jul 05 11:56:13.117461 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:13.117072 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerDied","Data":"22eb87a6f6860ab39c4f72b6310338ee7936235405b8414189f74995fc750c16"} Jul 05 11:56:13.117461 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:13.117157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"78d27f21e3686c12a0a66c03637e9fc9b8b7a633c488d3efb8d3b7ae9bcb4382"} Jul 05 11:56:14.124960 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.124924 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"00020586c2d03677c7d236da171df3abc54923b31177f18a990d1ccd1fc89514"} Jul 05 11:56:14.125437 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.124989 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"775427d034365d57302f2065b04457b1ffc16aea3bf8c5345150431cf308f6a7"} Jul 05 11:56:14.125437 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.125016 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"0f47bf2a4acbe6bdfeb73b4e0d815b7a96c069d101524b0464c141c9268cf02a"} Jul 05 11:56:14.125437 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.125040 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"7809a1ab7859081a035b80b50db3c778a62933bfdd76d52dcacc9175c8bbb3fc"} Jul 05 11:56:14.125437 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.125064 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"ca13652fe16f5e33395ac7904696a3cc1de58d9c051a0ad5f3776902f44ec2bb"} Jul 05 11:56:14.125437 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.125086 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5db51df0-29ab-4ecb-b550-ebca09ba4eb6","Type":"ContainerStarted","Data":"c3b0fbb4e4126617e35600917612b77baff7d3c9b83d014214f2b482dbc3682a"} Jul 05 11:56:14.151545 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:14.151502 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.130270385 podStartE2EDuration="2.151483751s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.213366ms" isStatefulPod=false podCreationTimestamp="2026-07-05 11:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:56:14.149935125 +0000 UTC m=+158.408145606" watchObservedRunningTime="2026-07-05 11:56:14.151483751 +0000 UTC m=+158.409694213" Jul 05 11:56:17.478775 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:56:17.478730 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:57:12.478913 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:57:12.478876 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:57:12.493921 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:57:12.493897 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:57:13.341914 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:57:13.341890 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 05 11:58:36.145266 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:58:36.145237 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 05 11:59:14.177058 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.177019 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x2ppn"] Jul 05 11:59:14.179167 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.179148 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.182051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.182029 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pl82g\"" Jul 05 11:59:14.182051 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.182043 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 05 11:59:14.183116 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.183096 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 05 11:59:14.189229 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.189210 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x2ppn"] Jul 05 11:59:14.196304 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.196285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfc6k\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-kube-api-access-jfc6k\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.196389 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.196316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.297194 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.297171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfc6k\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-kube-api-access-jfc6k\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.297307 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.297200 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.306212 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.306193 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.306496 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.306476 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfc6k\" (UniqueName: \"kubernetes.io/projected/4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc-kube-api-access-jfc6k\") pod \"cert-manager-55b4d954d5-x2ppn\" (UID: \"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc\") " pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.498914 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.498846 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x2ppn" Jul 05 11:59:14.617259 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.617237 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x2ppn"] Jul 05 11:59:14.619618 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:59:14.619594 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4adc0ac8_2f87_4d4f_97d5_ca85ee5606dc.slice/crio-1a31507ca4bd9026d93332ec8061d420630f7717574c382ad8765fc783df7b0b WatchSource:0}: Error finding container 1a31507ca4bd9026d93332ec8061d420630f7717574c382ad8765fc783df7b0b: Status 404 returned error can't find the container with id 1a31507ca4bd9026d93332ec8061d420630f7717574c382ad8765fc783df7b0b Jul 05 11:59:14.621564 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.621543 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 11:59:14.728528 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:14.728499 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x2ppn" event={"ID":"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc","Type":"ContainerStarted","Data":"1a31507ca4bd9026d93332ec8061d420630f7717574c382ad8765fc783df7b0b"} Jul 05 11:59:17.742159 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:17.742129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x2ppn" event={"ID":"4adc0ac8-2f87-4d4f-97d5-ca85ee5606dc","Type":"ContainerStarted","Data":"665ed7405902ff2b872ba22a25e8340984499359229f5fdd5a9778f0e077f6a8"} Jul 05 11:59:17.757839 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:17.757796 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-x2ppn" podStartSLOduration=0.734274333 podStartE2EDuration="3.75778162s" totalImagesPullingTime="3.023507287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:59:17.757225528 +0000 UTC m=+342.015435980" watchObservedRunningTime="2026-07-05 11:59:17.75778162 +0000 UTC m=+342.015992080" Jul 05 11:59:50.543562 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.543525 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7"] Jul 05 11:59:50.547842 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.547820 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.550783 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.550740 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 05 11:59:50.551869 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.551846 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 05 11:59:50.551987 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.551902 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 05 11:59:50.552130 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.552117 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rnb7g\"" Jul 05 11:59:50.552935 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.552920 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 05 11:59:50.568300 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.568273 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7"] Jul 05 11:59:50.569110 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.569095 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 05 11:59:50.657695 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.657666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.657844 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.657715 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gs8n\" (UniqueName: \"kubernetes.io/projected/70457dea-54c4-4d30-b272-4dee1c66eb8b-kube-api-access-5gs8n\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.657844 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.657787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-metrics-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.657844 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.657816 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70457dea-54c4-4d30-b272-4dee1c66eb8b-manager-config\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.758345 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.758317 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gs8n\" (UniqueName: \"kubernetes.io/projected/70457dea-54c4-4d30-b272-4dee1c66eb8b-kube-api-access-5gs8n\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.758474 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.758356 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-metrics-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.758474 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.758379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70457dea-54c4-4d30-b272-4dee1c66eb8b-manager-config\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.758474 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.758419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.759134 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.759111 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/70457dea-54c4-4d30-b272-4dee1c66eb8b-manager-config\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.760667 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.760644 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.760745 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.760721 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/70457dea-54c4-4d30-b272-4dee1c66eb8b-metrics-cert\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.769431 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.769410 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gs8n\" (UniqueName: \"kubernetes.io/projected/70457dea-54c4-4d30-b272-4dee1c66eb8b-kube-api-access-5gs8n\") pod \"lws-controller-manager-58544b45f-5k7t7\" (UID: \"70457dea-54c4-4d30-b272-4dee1c66eb8b\") " pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.856956 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.856902 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:50.988876 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:50.988845 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7"] Jul 05 11:59:50.991888 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:59:50.991861 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70457dea_54c4_4d30_b272_4dee1c66eb8b.slice/crio-8835fa2bb1b5cc6c3ba072c50c1f404f1daf853a4f2b965c8e5ebc0696676cef WatchSource:0}: Error finding container 8835fa2bb1b5cc6c3ba072c50c1f404f1daf853a4f2b965c8e5ebc0696676cef: Status 404 returned error can't find the container with id 8835fa2bb1b5cc6c3ba072c50c1f404f1daf853a4f2b965c8e5ebc0696676cef Jul 05 11:59:51.680913 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.680885 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw"] Jul 05 11:59:51.683510 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.683494 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.686646 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.686623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 05 11:59:51.686805 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.686680 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ctxds\"" Jul 05 11:59:51.686805 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.686625 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 05 11:59:51.686805 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.686737 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 05 11:59:51.686805 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.686702 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 05 11:59:51.699918 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.699899 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw"] Jul 05 11:59:51.764791 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.764750 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.764908 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.764803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.764908 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.764820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.764908 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.764891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncp7k\" (UniqueName: \"kubernetes.io/projected/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-kube-api-access-ncp7k\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.765038 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.764947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.865846 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.865822 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.865986 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.865864 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.865986 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.865889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.865986 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.865938 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncp7k\" (UniqueName: \"kubernetes.io/projected/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-kube-api-access-ncp7k\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.865986 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.865973 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.866351 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.866323 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-manifests\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.867071 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.867046 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" event={"ID":"70457dea-54c4-4d30-b272-4dee1c66eb8b","Type":"ContainerStarted","Data":"8835fa2bb1b5cc6c3ba072c50c1f404f1daf853a4f2b965c8e5ebc0696676cef"} Jul 05 11:59:51.868245 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.868228 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-tmp\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.868357 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.868340 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.868460 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.868441 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-webhook-cert\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.874129 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.874111 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncp7k\" (UniqueName: \"kubernetes.io/projected/23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30-kube-api-access-ncp7k\") pod \"opendatahub-operator-controller-manager-59595c8d98-4fmgw\" (UID: \"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30\") " pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:51.993318 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:51.993246 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:52.120299 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:52.120273 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw"] Jul 05 11:59:52.122730 ip-10-0-143-202 kubenswrapper[2574]: W0705 11:59:52.122700 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23771e3a_e3e6_432a_b7ab_9e3c4ff8ac30.slice/crio-2de46abcf408676dfa41c9cbb1581be37b4481ad02f172219781fdb4ca4424c5 WatchSource:0}: Error finding container 2de46abcf408676dfa41c9cbb1581be37b4481ad02f172219781fdb4ca4424c5: Status 404 returned error can't find the container with id 2de46abcf408676dfa41c9cbb1581be37b4481ad02f172219781fdb4ca4424c5 Jul 05 11:59:52.872613 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:52.872574 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" event={"ID":"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30","Type":"ContainerStarted","Data":"2de46abcf408676dfa41c9cbb1581be37b4481ad02f172219781fdb4ca4424c5"} Jul 05 11:59:56.889949 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:56.889899 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" event={"ID":"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30","Type":"ContainerStarted","Data":"89d69daa773a74b5a8bec3bc0c7627613f11b8842422e4300a628097d5b29b06"} Jul 05 11:59:56.891406 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:56.891380 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" event={"ID":"70457dea-54c4-4d30-b272-4dee1c66eb8b","Type":"ContainerStarted","Data":"861679f7fd3270d0185496ae5685b5d9033d54517d62338f379ed7aeaaf4ed11"} Jul 05 11:59:56.891530 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:56.891504 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 11:59:56.925730 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:56.925686 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" podStartSLOduration=1.6687337850000001 podStartE2EDuration="6.925671628s" totalImagesPullingTime="5.256937843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:59:56.924433348 +0000 UTC m=+381.182643809" watchObservedRunningTime="2026-07-05 11:59:56.925671628 +0000 UTC m=+381.183882149" Jul 05 11:59:57.896873 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:57.896838 2574 generic.go:356] "Generic (PLEG): container finished" podID="23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30" containerID="89d69daa773a74b5a8bec3bc0c7627613f11b8842422e4300a628097d5b29b06" exitCode=0 Jul 05 11:59:57.897318 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:57.896924 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" event={"ID":"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30","Type":"ContainerDied","Data":"89d69daa773a74b5a8bec3bc0c7627613f11b8842422e4300a628097d5b29b06"} Jul 05 11:59:58.902670 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:58.902637 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" event={"ID":"23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30","Type":"ContainerStarted","Data":"a6f4cd03f48661912c81d277ddee1c140dbd4c1a54475af8f2d001f9948632fd"} Jul 05 11:59:58.903036 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:58.902787 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 11:59:58.923701 ip-10-0-143-202 kubenswrapper[2574]: I0705 11:59:58.923655 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" podStartSLOduration=2.409721096 podStartE2EDuration="7.923642196s" totalImagesPullingTime="4.418663356s" totalInitContainerRuntime="1.095257744s" isStatefulPod=false podCreationTimestamp="2026-07-05 11:59:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 11:59:58.921972224 +0000 UTC m=+383.180182699" watchObservedRunningTime="2026-07-05 11:59:58.923642196 +0000 UTC m=+383.181852737" Jul 05 12:00:07.899239 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:07.899206 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58544b45f-5k7t7" Jul 05 12:00:09.909214 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:09.909182 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59595c8d98-4fmgw" Jul 05 12:00:12.013258 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.013222 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq"] Jul 05 12:00:12.070025 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.069984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq"] Jul 05 12:00:12.070183 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.070120 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.074288 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.074266 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-57829\"" Jul 05 12:00:12.074541 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.074522 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 05 12:00:12.074612 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.074596 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 05 12:00:12.075313 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.075297 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 05 12:00:12.075381 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.075361 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 05 12:00:12.228750 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.228708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837c4af3-92b0-44f9-883d-85668cd234c1-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.228927 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.228859 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x95w\" (UniqueName: \"kubernetes.io/projected/837c4af3-92b0-44f9-883d-85668cd234c1-kube-api-access-7x95w\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.228983 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.228931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/837c4af3-92b0-44f9-883d-85668cd234c1-tmp\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.330149 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.330067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x95w\" (UniqueName: \"kubernetes.io/projected/837c4af3-92b0-44f9-883d-85668cd234c1-kube-api-access-7x95w\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.330149 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.330119 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/837c4af3-92b0-44f9-883d-85668cd234c1-tmp\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.330149 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.330149 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837c4af3-92b0-44f9-883d-85668cd234c1-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.332435 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.332407 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/837c4af3-92b0-44f9-883d-85668cd234c1-tmp\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.332545 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.332523 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837c4af3-92b0-44f9-883d-85668cd234c1-tls-certs\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.339006 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.338979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x95w\" (UniqueName: \"kubernetes.io/projected/837c4af3-92b0-44f9-883d-85668cd234c1-kube-api-access-7x95w\") pod \"kube-auth-proxy-5f99dd8ff-gtnpq\" (UID: \"837c4af3-92b0-44f9-883d-85668cd234c1\") " pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.380177 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.380135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" Jul 05 12:00:12.542354 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.542328 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq"] Jul 05 12:00:12.544808 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:00:12.544739 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod837c4af3_92b0_44f9_883d_85668cd234c1.slice/crio-80edd95babb805c59a549857d3edf382baf4adb84970091dd385c287072de038 WatchSource:0}: Error finding container 80edd95babb805c59a549857d3edf382baf4adb84970091dd385c287072de038: Status 404 returned error can't find the container with id 80edd95babb805c59a549857d3edf382baf4adb84970091dd385c287072de038 Jul 05 12:00:12.961966 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:12.961928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" event={"ID":"837c4af3-92b0-44f9-883d-85668cd234c1","Type":"ContainerStarted","Data":"80edd95babb805c59a549857d3edf382baf4adb84970091dd385c287072de038"} Jul 05 12:00:15.975666 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:15.975630 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" event={"ID":"837c4af3-92b0-44f9-883d-85668cd234c1","Type":"ContainerStarted","Data":"e2d1a3e9b00ac6fcd3ce49a75e364f18236c2f4cb76b1a5d74340a311731d48f"} Jul 05 12:00:15.993011 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:00:15.992957 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f99dd8ff-gtnpq" podStartSLOduration=2.390357225 podStartE2EDuration="4.992941407s" totalImagesPullingTime="2.602584182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:00:15.99143127 +0000 UTC m=+400.249641743" watchObservedRunningTime="2026-07-05 12:00:15.992941407 +0000 UTC m=+400.251151866" Jul 05 12:01:17.033051 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.033011 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55899fd9dd-m9lhk"] Jul 05 12:01:17.036478 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.036452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040173 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040199 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040173 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040232 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040183 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tz7xq\"" Jul 05 12:01:17.040328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.040326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 05 12:01:17.044105 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.044082 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 05 12:01:17.046575 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.046554 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55899fd9dd-m9lhk"] Jul 05 12:01:17.161267 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-oauth-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161267 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161274 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-oauth-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwgv8\" (UniqueName: \"kubernetes.io/projected/a5b27eff-3c6d-4472-8009-2acb2e584f6d-kube-api-access-jwgv8\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161408 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-trusted-ca-bundle\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-service-ca\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.161547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.161533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262120 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262075 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwgv8\" (UniqueName: \"kubernetes.io/projected/a5b27eff-3c6d-4472-8009-2acb2e584f6d-kube-api-access-jwgv8\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262135 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-trusted-ca-bundle\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262201 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-service-ca\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-oauth-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.262319 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.262296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-oauth-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.263131 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.263100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-oauth-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.263272 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.263135 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.263272 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.263166 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-service-ca\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.263371 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.263298 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5b27eff-3c6d-4472-8009-2acb2e584f6d-trusted-ca-bundle\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.264812 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.264781 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-oauth-config\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.265111 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.265092 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5b27eff-3c6d-4472-8009-2acb2e584f6d-console-serving-cert\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.269880 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.269858 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwgv8\" (UniqueName: \"kubernetes.io/projected/a5b27eff-3c6d-4472-8009-2acb2e584f6d-kube-api-access-jwgv8\") pod \"console-55899fd9dd-m9lhk\" (UID: \"a5b27eff-3c6d-4472-8009-2acb2e584f6d\") " pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.347013 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.346908 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:17.485090 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:17.485059 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55899fd9dd-m9lhk"] Jul 05 12:01:17.487459 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:01:17.487430 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b27eff_3c6d_4472_8009_2acb2e584f6d.slice/crio-506b14e2252e7927242f539dd2a7d8b3d36c0d5c6f77c2046f22c88bf88a58c7 WatchSource:0}: Error finding container 506b14e2252e7927242f539dd2a7d8b3d36c0d5c6f77c2046f22c88bf88a58c7: Status 404 returned error can't find the container with id 506b14e2252e7927242f539dd2a7d8b3d36c0d5c6f77c2046f22c88bf88a58c7 Jul 05 12:01:18.207059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:18.207024 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55899fd9dd-m9lhk" event={"ID":"a5b27eff-3c6d-4472-8009-2acb2e584f6d","Type":"ContainerStarted","Data":"ef2641d5987738c833c44c5b9a583b82d6f11de0c412b3717547abd081dd8bb8"} Jul 05 12:01:18.207444 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:18.207102 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55899fd9dd-m9lhk" event={"ID":"a5b27eff-3c6d-4472-8009-2acb2e584f6d","Type":"ContainerStarted","Data":"506b14e2252e7927242f539dd2a7d8b3d36c0d5c6f77c2046f22c88bf88a58c7"} Jul 05 12:01:18.225217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:18.225165 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55899fd9dd-m9lhk" podStartSLOduration=1.225152593 podStartE2EDuration="1.225152593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:01:18.22386844 +0000 UTC m=+462.482078915" watchObservedRunningTime="2026-07-05 12:01:18.225152593 +0000 UTC m=+462.483363052" Jul 05 12:01:27.348188 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:27.348141 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:27.348188 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:27.348202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:27.352914 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:27.352891 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:01:28.250190 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:01:28.250162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55899fd9dd-m9lhk" Jul 05 12:02:05.656922 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.656885 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k"] Jul 05 12:02:05.660192 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.660175 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.662606 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.662575 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 05 12:02:05.662730 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.662711 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 05 12:02:05.662977 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.662959 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 05 12:02:05.663747 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.663728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 05 12:02:05.663904 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.663887 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r5gq7\"" Jul 05 12:02:05.670682 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.670657 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k"] Jul 05 12:02:05.779627 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.779591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.779859 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.779650 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.779859 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.779754 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z95cd\" (UniqueName: \"kubernetes.io/projected/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-kube-api-access-z95cd\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.880963 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.880923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.881153 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.880983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z95cd\" (UniqueName: \"kubernetes.io/projected/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-kube-api-access-z95cd\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.881153 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.881023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.881153 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:05.881117 2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 05 12:02:05.881315 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:05.881172 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert podName:5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244 nodeName:}" failed. No retries permitted until 2026-07-05 12:02:06.381156093 +0000 UTC m=+510.639366531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-hkf4k" (UID: "5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244") : secret "plugin-serving-cert" not found Jul 05 12:02:05.881663 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.881635 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:05.889271 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:05.889245 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z95cd\" (UniqueName: \"kubernetes.io/projected/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-kube-api-access-z95cd\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:06.385219 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:06.385186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:06.387590 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:06.387558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hkf4k\" (UID: \"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:06.569948 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:06.569897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" Jul 05 12:02:06.693785 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:06.693745 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k"] Jul 05 12:02:06.696327 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:02:06.696299 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5adeb6f1_eee2_4f58_ad0d_3b1ea3fab244.slice/crio-987d5f57f94d7bf198975687e316ab9da430597e18283fde205e83d59682c897 WatchSource:0}: Error finding container 987d5f57f94d7bf198975687e316ab9da430597e18283fde205e83d59682c897: Status 404 returned error can't find the container with id 987d5f57f94d7bf198975687e316ab9da430597e18283fde205e83d59682c897 Jul 05 12:02:07.401095 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:07.401056 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" event={"ID":"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244","Type":"ContainerStarted","Data":"987d5f57f94d7bf198975687e316ab9da430597e18283fde205e83d59682c897"} Jul 05 12:02:15.566900 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.566861 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs"] Jul 05 12:02:15.577156 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.577124 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.580445 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.580414 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r799m\"" Jul 05 12:02:15.586199 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.586159 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs"] Jul 05 12:02:15.661754 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.661720 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx27h\" (UniqueName: \"kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.661963 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.661877 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.762878 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.762834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.763050 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.762932 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx27h\" (UniqueName: \"kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.763314 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.763283 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.774071 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.774044 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx27h\" (UniqueName: \"kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:15.890059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:15.889978 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:16.281659 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.281620 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs"] Jul 05 12:02:16.284663 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.284632 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs"] Jul 05 12:02:16.298308 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.298277 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:02:16.313432 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.313376 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.314807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.314781 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:02:16.369275 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.369238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f47rg\" (UniqueName: \"kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.369458 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.369342 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.469951 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.469899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f47rg\" (UniqueName: \"kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.470139 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.470009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.470436 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.470412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.480074 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.480050 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f47rg\" (UniqueName: \"kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-bmmzp\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:16.627046 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:16.626964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:29.596446 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:29.596420 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:02:29.598970 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:02:29.598937 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a61f6d_9ab0_4a07_b536_1c5de8229848.slice/crio-99c30c11f58ea72c790a2a839990229eb77a9793735bbca367f9f2973991789f WatchSource:0}: Error finding container 99c30c11f58ea72c790a2a839990229eb77a9793735bbca367f9f2973991789f: Status 404 returned error can't find the container with id 99c30c11f58ea72c790a2a839990229eb77a9793735bbca367f9f2973991789f Jul 05 12:02:30.514844 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:30.514810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" event={"ID":"b5a61f6d-9ab0-4a07-b536-1c5de8229848","Type":"ContainerStarted","Data":"99c30c11f58ea72c790a2a839990229eb77a9793735bbca367f9f2973991789f"} Jul 05 12:02:30.516217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:30.516190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" event={"ID":"5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244","Type":"ContainerStarted","Data":"fc0aa46287e7a6e82962b81a8ee532541a65658cede9455fc2493ffe6c8a6f98"} Jul 05 12:02:30.538053 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:30.538002 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hkf4k" podStartSLOduration=2.204016557 podStartE2EDuration="25.537986687s" totalImagesPullingTime="23.33397013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:02:30.535497076 +0000 UTC m=+534.793707537" watchObservedRunningTime="2026-07-05 12:02:30.537986687 +0000 UTC m=+534.796197147" Jul 05 12:02:32.010993 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:32.010939 2574 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-dxmxs_kuadrant-system_0e59df2a-491f-4252-8995-31ae5885938c_0(e93bbe3f82b4b18382a52e97277503481d365b4e7b001ca21ba1424b1a201b25): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-dxmxs to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 05 12:02:32.011461 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:32.011024 2574 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-dxmxs_kuadrant-system_0e59df2a-491f-4252-8995-31ae5885938c_0(e93bbe3f82b4b18382a52e97277503481d365b4e7b001ca21ba1424b1a201b25): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-dxmxs to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:32.528868 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.528836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:32.531128 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:32.531093 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" is forbidden: User \"system:node:ip-10-0-143-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-202.ec2.internal' and this object" podUID="0e59df2a-491f-4252-8995-31ae5885938c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:32.536018 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.535994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:32.538047 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:32.538009 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" is forbidden: User \"system:node:ip-10-0-143-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-202.ec2.internal' and this object" podUID="0e59df2a-491f-4252-8995-31ae5885938c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:32.619695 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.619661 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume\") pod \"0e59df2a-491f-4252-8995-31ae5885938c\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " Jul 05 12:02:32.620126 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.619727 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx27h\" (UniqueName: \"kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h\") pod \"0e59df2a-491f-4252-8995-31ae5885938c\" (UID: \"0e59df2a-491f-4252-8995-31ae5885938c\") " Jul 05 12:02:32.620126 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.620017 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0e59df2a-491f-4252-8995-31ae5885938c" (UID: "0e59df2a-491f-4252-8995-31ae5885938c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:02:32.622148 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.622114 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h" (OuterVolumeSpecName: "kube-api-access-vx27h") pod "0e59df2a-491f-4252-8995-31ae5885938c" (UID: "0e59df2a-491f-4252-8995-31ae5885938c"). InnerVolumeSpecName "kube-api-access-vx27h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:02:32.721459 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.721418 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx27h\" (UniqueName: \"kubernetes.io/projected/0e59df2a-491f-4252-8995-31ae5885938c-kube-api-access-vx27h\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:02:32.721459 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:32.721459 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e59df2a-491f-4252-8995-31ae5885938c-extensions-socket-volume\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:02:33.533693 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:33.533660 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:33.536130 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:33.536097 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" is forbidden: User \"system:node:ip-10-0-143-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-202.ec2.internal' and this object" podUID="0e59df2a-491f-4252-8995-31ae5885938c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:33.542817 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:33.542782 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-dxmxs\" is forbidden: User \"system:node:ip-10-0-143-202.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-202.ec2.internal' and this object" podUID="0e59df2a-491f-4252-8995-31ae5885938c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-dxmxs" Jul 05 12:02:34.245116 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:34.245080 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e59df2a-491f-4252-8995-31ae5885938c" path="/var/lib/kubelet/pods/0e59df2a-491f-4252-8995-31ae5885938c/volumes" Jul 05 12:02:35.547041 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:35.546998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" event={"ID":"b5a61f6d-9ab0-4a07-b536-1c5de8229848","Type":"ContainerStarted","Data":"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a"} Jul 05 12:02:35.547560 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:35.547141 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:35.564801 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:35.564727 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" podStartSLOduration=14.504686124 podStartE2EDuration="19.564714941s" totalImagesPullingTime="5.060028817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:02:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:02:35.56454111 +0000 UTC m=+539.822751593" watchObservedRunningTime="2026-07-05 12:02:35.564714941 +0000 UTC m=+539.822925400" Jul 05 12:02:46.553586 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:46.553552 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:58.933499 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:58.933460 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:02:58.933944 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:58.933662 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" podUID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" containerName="manager" containerID="cri-o://6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a" gracePeriod=10 Jul 05 12:02:59.177899 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.177876 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:59.353044 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.352957 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f47rg\" (UniqueName: \"kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg\") pod \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " Jul 05 12:02:59.353044 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.353013 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume\") pod \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\" (UID: \"b5a61f6d-9ab0-4a07-b536-1c5de8229848\") " Jul 05 12:02:59.353425 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.353404 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b5a61f6d-9ab0-4a07-b536-1c5de8229848" (UID: "b5a61f6d-9ab0-4a07-b536-1c5de8229848"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 05 12:02:59.355201 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.355178 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg" (OuterVolumeSpecName: "kube-api-access-f47rg") pod "b5a61f6d-9ab0-4a07-b536-1c5de8229848" (UID: "b5a61f6d-9ab0-4a07-b536-1c5de8229848"). InnerVolumeSpecName "kube-api-access-f47rg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:02:59.453586 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.453555 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f47rg\" (UniqueName: \"kubernetes.io/projected/b5a61f6d-9ab0-4a07-b536-1c5de8229848-kube-api-access-f47rg\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:02:59.453586 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.453580 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5a61f6d-9ab0-4a07-b536-1c5de8229848-extensions-socket-volume\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:02:59.644992 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.644909 2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" containerID="6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a" exitCode=0 Jul 05 12:02:59.644992 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.644957 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" Jul 05 12:02:59.644992 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.644969 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" event={"ID":"b5a61f6d-9ab0-4a07-b536-1c5de8229848","Type":"ContainerDied","Data":"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a"} Jul 05 12:02:59.645190 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.645022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp" event={"ID":"b5a61f6d-9ab0-4a07-b536-1c5de8229848","Type":"ContainerDied","Data":"99c30c11f58ea72c790a2a839990229eb77a9793735bbca367f9f2973991789f"} Jul 05 12:02:59.645190 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.645053 2574 scope.go:117] "RemoveContainer" containerID="6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a" Jul 05 12:02:59.654069 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.654050 2574 scope.go:117] "RemoveContainer" containerID="6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a" Jul 05 12:02:59.654369 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:02:59.654337 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a\": container with ID starting with 6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a not found: ID does not exist" containerID="6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a" Jul 05 12:02:59.654462 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.654378 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a"} err="failed to get container status \"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a\": rpc error: code = NotFound desc = could not find container \"6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a\": container with ID starting with 6bd99cccc487464f0d2bbd8b2810f3119f819078a21b657c2f1d9e0ffd50713a not found: ID does not exist" Jul 05 12:02:59.668517 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.668491 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:02:59.675422 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:02:59.675401 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-bmmzp"] Jul 05 12:03:00.244220 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:00.244187 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" path="/var/lib/kubelet/pods/b5a61f6d-9ab0-4a07-b536-1c5de8229848/volumes" Jul 05 12:03:07.132149 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.132112 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:07.132655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.132414 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" containerName="manager" Jul 05 12:03:07.132655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.132425 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" containerName="manager" Jul 05 12:03:07.132655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.132502 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a61f6d-9ab0-4a07-b536-1c5de8229848" containerName="manager" Jul 05 12:03:07.170514 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.170473 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:07.170671 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.170583 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.172969 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.172946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 05 12:03:07.211663 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.211627 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wnp\" (UniqueName: \"kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.211875 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.211686 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.230895 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.230856 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:07.312543 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.312503 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wnp\" (UniqueName: \"kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.312543 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.312546 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.313191 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.313173 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.324815 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.324791 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wnp\" (UniqueName: \"kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp\") pod \"limitador-limitador-64c94fd64f-bj6l6\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.480445 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.480416 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:07.625516 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.625369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:07.628435 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:07.628403 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a30ef0_dbe5_4210_81fb_657e9e6c8c9c.slice/crio-01b9a47c8f650674d81cb7c01b620d3aba7bf92ba1019200d2999aac1d7aa564 WatchSource:0}: Error finding container 01b9a47c8f650674d81cb7c01b620d3aba7bf92ba1019200d2999aac1d7aa564: Status 404 returned error can't find the container with id 01b9a47c8f650674d81cb7c01b620d3aba7bf92ba1019200d2999aac1d7aa564 Jul 05 12:03:07.686610 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.686572 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" event={"ID":"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c","Type":"ContainerStarted","Data":"01b9a47c8f650674d81cb7c01b620d3aba7bf92ba1019200d2999aac1d7aa564"} Jul 05 12:03:07.912289 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.912207 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:07.944299 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.944264 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:07.944446 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.944360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:07.946969 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:07.946944 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-trc49\"" Jul 05 12:03:08.016070 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.016036 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:08.018847 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.018824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9rbl\" (UniqueName: \"kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl\") pod \"authorino-58cd955675-fxzfs\" (UID: \"3dc06ed2-f340-4ff4-8855-97d4b35b067b\") " pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:08.035636 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.035606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:08.035800 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.035701 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:08.119647 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.119610 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfvrn\" (UniqueName: \"kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn\") pod \"authorino-75d556bd77-fkc9m\" (UID: \"1e138f58-6cd8-4741-85dc-088872c729e1\") " pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:08.119843 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.119660 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9rbl\" (UniqueName: \"kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl\") pod \"authorino-58cd955675-fxzfs\" (UID: \"3dc06ed2-f340-4ff4-8855-97d4b35b067b\") " pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:08.127424 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.127394 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9rbl\" (UniqueName: \"kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl\") pod \"authorino-58cd955675-fxzfs\" (UID: \"3dc06ed2-f340-4ff4-8855-97d4b35b067b\") " pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:08.220225 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.220181 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfvrn\" (UniqueName: \"kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn\") pod \"authorino-75d556bd77-fkc9m\" (UID: \"1e138f58-6cd8-4741-85dc-088872c729e1\") " pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:08.229461 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.229428 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfvrn\" (UniqueName: \"kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn\") pod \"authorino-75d556bd77-fkc9m\" (UID: \"1e138f58-6cd8-4741-85dc-088872c729e1\") " pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:08.253850 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.253819 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:08.344831 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.344745 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:08.414362 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.414331 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:08.417127 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:08.417091 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dc06ed2_f340_4ff4_8855_97d4b35b067b.slice/crio-39db329b2c18e44d54b2b285c97726ff4cd1ca124ae6bf465f5f46bb77b18995 WatchSource:0}: Error finding container 39db329b2c18e44d54b2b285c97726ff4cd1ca124ae6bf465f5f46bb77b18995: Status 404 returned error can't find the container with id 39db329b2c18e44d54b2b285c97726ff4cd1ca124ae6bf465f5f46bb77b18995 Jul 05 12:03:08.534261 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.534230 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:08.537794 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:08.537738 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e138f58_6cd8_4741_85dc_088872c729e1.slice/crio-e29c1320c3ecdedc6c330e7a438b863d03e55d347e5a72cdf168e2b1c27f85c5 WatchSource:0}: Error finding container e29c1320c3ecdedc6c330e7a438b863d03e55d347e5a72cdf168e2b1c27f85c5: Status 404 returned error can't find the container with id e29c1320c3ecdedc6c330e7a438b863d03e55d347e5a72cdf168e2b1c27f85c5 Jul 05 12:03:08.693836 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.693795 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fxzfs" event={"ID":"3dc06ed2-f340-4ff4-8855-97d4b35b067b","Type":"ContainerStarted","Data":"39db329b2c18e44d54b2b285c97726ff4cd1ca124ae6bf465f5f46bb77b18995"} Jul 05 12:03:08.695387 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:08.695357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkc9m" event={"ID":"1e138f58-6cd8-4741-85dc-088872c729e1","Type":"ContainerStarted","Data":"e29c1320c3ecdedc6c330e7a438b863d03e55d347e5a72cdf168e2b1c27f85c5"} Jul 05 12:03:13.722383 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.722347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkc9m" event={"ID":"1e138f58-6cd8-4741-85dc-088872c729e1","Type":"ContainerStarted","Data":"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071"} Jul 05 12:03:13.723912 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.723884 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fxzfs" event={"ID":"3dc06ed2-f340-4ff4-8855-97d4b35b067b","Type":"ContainerStarted","Data":"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54"} Jul 05 12:03:13.725283 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.725260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" event={"ID":"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c","Type":"ContainerStarted","Data":"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556"} Jul 05 12:03:13.725378 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.725340 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:13.737103 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.737056 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-fkc9m" podStartSLOduration=1.357023114 podStartE2EDuration="5.737042419s" totalImagesPullingTime="4.380019305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:13.735651601 +0000 UTC m=+577.993862061" watchObservedRunningTime="2026-07-05 12:03:13.737042419 +0000 UTC m=+577.995252857" Jul 05 12:03:13.751466 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.751417 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" podStartSLOduration=1.453820312 podStartE2EDuration="6.751401747s" totalImagesPullingTime="5.297581435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:13.750424196 +0000 UTC m=+578.008634654" watchObservedRunningTime="2026-07-05 12:03:13.751401747 +0000 UTC m=+578.009612206" Jul 05 12:03:13.760775 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.760730 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:13.773661 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:13.773606 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-fxzfs" podStartSLOduration=2.260046002 podStartE2EDuration="6.773589389s" totalImagesPullingTime="4.513543387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:13.771969602 +0000 UTC m=+578.030180063" watchObservedRunningTime="2026-07-05 12:03:13.773589389 +0000 UTC m=+578.031799848" Jul 05 12:03:15.733140 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:15.733101 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-fxzfs" podUID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" containerName="authorino" containerID="cri-o://fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54" gracePeriod=30 Jul 05 12:03:15.977077 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:15.977047 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:16.094751 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.094714 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9rbl\" (UniqueName: \"kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl\") pod \"3dc06ed2-f340-4ff4-8855-97d4b35b067b\" (UID: \"3dc06ed2-f340-4ff4-8855-97d4b35b067b\") " Jul 05 12:03:16.096754 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.096727 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl" (OuterVolumeSpecName: "kube-api-access-f9rbl") pod "3dc06ed2-f340-4ff4-8855-97d4b35b067b" (UID: "3dc06ed2-f340-4ff4-8855-97d4b35b067b"). InnerVolumeSpecName "kube-api-access-f9rbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:03:16.196023 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.195988 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9rbl\" (UniqueName: \"kubernetes.io/projected/3dc06ed2-f340-4ff4-8855-97d4b35b067b-kube-api-access-f9rbl\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:16.738894 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.738856 2574 generic.go:356] "Generic (PLEG): container finished" podID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" containerID="fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54" exitCode=0 Jul 05 12:03:16.738894 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.738896 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-fxzfs" Jul 05 12:03:16.739430 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.738949 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fxzfs" event={"ID":"3dc06ed2-f340-4ff4-8855-97d4b35b067b","Type":"ContainerDied","Data":"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54"} Jul 05 12:03:16.739430 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.738993 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-fxzfs" event={"ID":"3dc06ed2-f340-4ff4-8855-97d4b35b067b","Type":"ContainerDied","Data":"39db329b2c18e44d54b2b285c97726ff4cd1ca124ae6bf465f5f46bb77b18995"} Jul 05 12:03:16.739430 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.739020 2574 scope.go:117] "RemoveContainer" containerID="fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54" Jul 05 12:03:16.747182 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.747165 2574 scope.go:117] "RemoveContainer" containerID="fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54" Jul 05 12:03:16.747450 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:16.747433 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54\": container with ID starting with fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54 not found: ID does not exist" containerID="fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54" Jul 05 12:03:16.747489 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.747461 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54"} err="failed to get container status \"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54\": rpc error: code = NotFound desc = could not find container \"fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54\": container with ID starting with fb02531bcbdf4bad3b0e27c04722fa613c294ceef713000341d7fa03059f5a54 not found: ID does not exist" Jul 05 12:03:16.753679 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.753656 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:16.757674 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:16.757652 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-fxzfs"] Jul 05 12:03:18.244239 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:18.244208 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" path="/var/lib/kubelet/pods/3dc06ed2-f340-4ff4-8855-97d4b35b067b/volumes" Jul 05 12:03:22.036276 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.036241 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:22.036694 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.036465 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" podUID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" containerName="limitador" containerID="cri-o://fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556" gracePeriod=30 Jul 05 12:03:22.038475 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.038449 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:22.585739 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.585710 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:22.651559 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.651481 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file\") pod \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " Jul 05 12:03:22.651559 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.651550 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7wnp\" (UniqueName: \"kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp\") pod \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\" (UID: \"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c\") " Jul 05 12:03:22.651858 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.651833 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file" (OuterVolumeSpecName: "config-file") pod "c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" (UID: "c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:03:22.653526 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.653499 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp" (OuterVolumeSpecName: "kube-api-access-t7wnp") pod "c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" (UID: "c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c"). InnerVolumeSpecName "kube-api-access-t7wnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:03:22.753099 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.753061 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7wnp\" (UniqueName: \"kubernetes.io/projected/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-kube-api-access-t7wnp\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:22.753099 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.753092 2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c-config-file\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:22.766042 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.766010 2574 generic.go:356] "Generic (PLEG): container finished" podID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" containerID="fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556" exitCode=0 Jul 05 12:03:22.766042 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.766042 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" Jul 05 12:03:22.766278 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.766073 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" event={"ID":"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c","Type":"ContainerDied","Data":"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556"} Jul 05 12:03:22.766278 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.766133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bj6l6" event={"ID":"c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c","Type":"ContainerDied","Data":"01b9a47c8f650674d81cb7c01b620d3aba7bf92ba1019200d2999aac1d7aa564"} Jul 05 12:03:22.766278 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.766164 2574 scope.go:117] "RemoveContainer" containerID="fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556" Jul 05 12:03:22.774417 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.774400 2574 scope.go:117] "RemoveContainer" containerID="fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556" Jul 05 12:03:22.774689 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:22.774668 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556\": container with ID starting with fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556 not found: ID does not exist" containerID="fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556" Jul 05 12:03:22.774755 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.774697 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556"} err="failed to get container status \"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556\": rpc error: code = NotFound desc = could not find container \"fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556\": container with ID starting with fd56154b2e0b3a2c9ff88e745c4624b1d05743bc1f6c7128f71a5a7b2f557556 not found: ID does not exist" Jul 05 12:03:22.786391 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.786367 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:22.789395 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:22.789368 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bj6l6"] Jul 05 12:03:24.243733 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:24.243700 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" path="/var/lib/kubelet/pods/c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c/volumes" Jul 05 12:03:26.900354 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900309 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mcbw5"] Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900611 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" containerName="authorino" Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900622 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" containerName="authorino" Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900644 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" containerName="limitador" Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900650 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" containerName="limitador" Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900714 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2a30ef0-dbe5-4210-81fb-657e9e6c8c9c" containerName="limitador" Jul 05 12:03:26.900807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.900723 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc06ed2-f340-4ff4-8855-97d4b35b067b" containerName="authorino" Jul 05 12:03:26.904955 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.904933 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:26.907712 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.907689 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 05 12:03:26.908004 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.907985 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vqpt4\"" Jul 05 12:03:26.910796 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.910733 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mcbw5"] Jul 05 12:03:26.991200 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.991161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77ddbe0f-b7d9-469e-bf28-b65db9212a33-data\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:26.991338 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:26.991234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stvkp\" (UniqueName: \"kubernetes.io/projected/77ddbe0f-b7d9-469e-bf28-b65db9212a33-kube-api-access-stvkp\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.091623 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.091583 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77ddbe0f-b7d9-469e-bf28-b65db9212a33-data\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.091816 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.091687 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stvkp\" (UniqueName: \"kubernetes.io/projected/77ddbe0f-b7d9-469e-bf28-b65db9212a33-kube-api-access-stvkp\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.092034 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.092002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/77ddbe0f-b7d9-469e-bf28-b65db9212a33-data\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.099578 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.099553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stvkp\" (UniqueName: \"kubernetes.io/projected/77ddbe0f-b7d9-469e-bf28-b65db9212a33-kube-api-access-stvkp\") pod \"postgres-69fb4594cf-mcbw5\" (UID: \"77ddbe0f-b7d9-469e-bf28-b65db9212a33\") " pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.217453 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.217417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:27.342870 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.342844 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mcbw5"] Jul 05 12:03:27.345309 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:27.345288 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ddbe0f_b7d9_469e_bf28_b65db9212a33.slice/crio-daf453e443143e418143a8b7ff15d151a1f887fe653df1e7019bdf66f30bf6cf WatchSource:0}: Error finding container daf453e443143e418143a8b7ff15d151a1f887fe653df1e7019bdf66f30bf6cf: Status 404 returned error can't find the container with id daf453e443143e418143a8b7ff15d151a1f887fe653df1e7019bdf66f30bf6cf Jul 05 12:03:27.788169 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:27.788133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mcbw5" event={"ID":"77ddbe0f-b7d9-469e-bf28-b65db9212a33","Type":"ContainerStarted","Data":"daf453e443143e418143a8b7ff15d151a1f887fe653df1e7019bdf66f30bf6cf"} Jul 05 12:03:32.814445 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:32.814354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mcbw5" event={"ID":"77ddbe0f-b7d9-469e-bf28-b65db9212a33","Type":"ContainerStarted","Data":"4de92c620b018ce6ce19aa014a2e2c600fba7125efef6224fa091ce3339f5e3f"} Jul 05 12:03:32.814832 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:32.814520 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:32.829409 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:32.829341 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mcbw5" podStartSLOduration=1.6642548270000002 podStartE2EDuration="6.829319599s" totalImagesPullingTime="5.165064772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:32.828310032 +0000 UTC m=+597.086520561" watchObservedRunningTime="2026-07-05 12:03:32.829319599 +0000 UTC m=+597.087530060" Jul 05 12:03:38.851089 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:38.851064 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mcbw5" Jul 05 12:03:39.385137 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.385102 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:39.390515 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.390497 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:39.396532 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.396500 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:39.503475 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.503435 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdkbp\" (UniqueName: \"kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp\") pod \"authorino-5d4b67c446-jdlgt\" (UID: \"4c0e9405-2457-4dff-bffc-123dce2c6cae\") " pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:39.604047 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.604005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdkbp\" (UniqueName: \"kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp\") pod \"authorino-5d4b67c446-jdlgt\" (UID: \"4c0e9405-2457-4dff-bffc-123dce2c6cae\") " pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:39.615182 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.615151 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdkbp\" (UniqueName: \"kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp\") pod \"authorino-5d4b67c446-jdlgt\" (UID: \"4c0e9405-2457-4dff-bffc-123dce2c6cae\") " pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:39.622180 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.622153 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:39.622392 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.622381 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:39.646923 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.646890 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:39.651538 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.651513 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:39.656622 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.656597 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:39.753353 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.753328 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:39.755909 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:39.755883 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c0e9405_2457_4dff_bffc_123dce2c6cae.slice/crio-4c8a77038cc50b97f3082212c5952e99ff798bf5c92b3ba5bb0a1135c2075ff4 WatchSource:0}: Error finding container 4c8a77038cc50b97f3082212c5952e99ff798bf5c92b3ba5bb0a1135c2075ff4: Status 404 returned error can't find the container with id 4c8a77038cc50b97f3082212c5952e99ff798bf5c92b3ba5bb0a1135c2075ff4 Jul 05 12:03:39.806003 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.805966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zglf\" (UniqueName: \"kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf\") pod \"authorino-6c9cdd7bcc-hfpvz\" (UID: \"3db85c56-fb5e-4ad1-98f7-5abb4877c051\") " pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:39.847208 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.847163 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" event={"ID":"4c0e9405-2457-4dff-bffc-123dce2c6cae","Type":"ContainerStarted","Data":"4c8a77038cc50b97f3082212c5952e99ff798bf5c92b3ba5bb0a1135c2075ff4"} Jul 05 12:03:39.907474 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.907376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zglf\" (UniqueName: \"kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf\") pod \"authorino-6c9cdd7bcc-hfpvz\" (UID: \"3db85c56-fb5e-4ad1-98f7-5abb4877c051\") " pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:39.915170 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.915144 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zglf\" (UniqueName: \"kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf\") pod \"authorino-6c9cdd7bcc-hfpvz\" (UID: \"3db85c56-fb5e-4ad1-98f7-5abb4877c051\") " pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:39.962645 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:39.962612 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:40.016798 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.016748 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:40.041712 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.041682 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:03:40.071536 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.071494 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:03:40.071676 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.071595 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.074199 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.074137 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 05 12:03:40.121160 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.121138 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:40.123098 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:40.123057 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3db85c56_fb5e_4ad1_98f7_5abb4877c051.slice/crio-5a225848c1134a28c1c6d3eb48c285df97c0a332d9236ca7f96b6f77429c799e WatchSource:0}: Error finding container 5a225848c1134a28c1c6d3eb48c285df97c0a332d9236ca7f96b6f77429c799e: Status 404 returned error can't find the container with id 5a225848c1134a28c1c6d3eb48c285df97c0a332d9236ca7f96b6f77429c799e Jul 05 12:03:40.210072 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.210034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.210251 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.210074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqjvr\" (UniqueName: \"kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.310714 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.310692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqjvr\" (UniqueName: \"kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.310862 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.310793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.313045 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.313024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.318421 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.318395 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqjvr\" (UniqueName: \"kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr\") pod \"authorino-69c5568fd9-cn59h\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.382347 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.382316 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:03:40.532264 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.532238 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:03:40.533905 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:40.533882 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf024e22a_6f4d_49be_acf1_929da1510822.slice/crio-0e0c150e2284b79701a6c665ed49ac8587169a9567595ea30a74f0597c0e32d6 WatchSource:0}: Error finding container 0e0c150e2284b79701a6c665ed49ac8587169a9567595ea30a74f0597c0e32d6: Status 404 returned error can't find the container with id 0e0c150e2284b79701a6c665ed49ac8587169a9567595ea30a74f0597c0e32d6 Jul 05 12:03:40.859095 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.859051 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" event={"ID":"4c0e9405-2457-4dff-bffc-123dce2c6cae","Type":"ContainerStarted","Data":"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0"} Jul 05 12:03:40.859427 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.859386 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" podUID="4c0e9405-2457-4dff-bffc-123dce2c6cae" containerName="authorino" containerID="cri-o://10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0" gracePeriod=30 Jul 05 12:03:40.860624 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.860565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c5568fd9-cn59h" event={"ID":"f024e22a-6f4d-49be-acf1-929da1510822","Type":"ContainerStarted","Data":"0e0c150e2284b79701a6c665ed49ac8587169a9567595ea30a74f0597c0e32d6"} Jul 05 12:03:40.862642 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.862607 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" event={"ID":"3db85c56-fb5e-4ad1-98f7-5abb4877c051","Type":"ContainerStarted","Data":"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d"} Jul 05 12:03:40.862790 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.862646 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" event={"ID":"3db85c56-fb5e-4ad1-98f7-5abb4877c051","Type":"ContainerStarted","Data":"5a225848c1134a28c1c6d3eb48c285df97c0a332d9236ca7f96b6f77429c799e"} Jul 05 12:03:40.862857 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.862786 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" podUID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" containerName="authorino" containerID="cri-o://e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d" gracePeriod=30 Jul 05 12:03:40.874957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.874912 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" podStartSLOduration=1.376202374 podStartE2EDuration="1.874897648s" totalImagesPullingTime="498.695274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:40.873061402 +0000 UTC m=+605.131271861" watchObservedRunningTime="2026-07-05 12:03:40.874897648 +0000 UTC m=+605.133108107" Jul 05 12:03:40.886541 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:40.886497 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" podStartSLOduration=1.32109387 podStartE2EDuration="1.886482152s" totalImagesPullingTime="565.388282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:40.88561908 +0000 UTC m=+605.143829539" watchObservedRunningTime="2026-07-05 12:03:40.886482152 +0000 UTC m=+605.144692653" Jul 05 12:03:41.131107 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.131088 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:41.168592 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.168569 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:41.319555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.319468 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdkbp\" (UniqueName: \"kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp\") pod \"4c0e9405-2457-4dff-bffc-123dce2c6cae\" (UID: \"4c0e9405-2457-4dff-bffc-123dce2c6cae\") " Jul 05 12:03:41.319555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.319531 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zglf\" (UniqueName: \"kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf\") pod \"3db85c56-fb5e-4ad1-98f7-5abb4877c051\" (UID: \"3db85c56-fb5e-4ad1-98f7-5abb4877c051\") " Jul 05 12:03:41.321540 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.321510 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp" (OuterVolumeSpecName: "kube-api-access-hdkbp") pod "4c0e9405-2457-4dff-bffc-123dce2c6cae" (UID: "4c0e9405-2457-4dff-bffc-123dce2c6cae"). InnerVolumeSpecName "kube-api-access-hdkbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:03:41.321540 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.321521 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf" (OuterVolumeSpecName: "kube-api-access-8zglf") pod "3db85c56-fb5e-4ad1-98f7-5abb4877c051" (UID: "3db85c56-fb5e-4ad1-98f7-5abb4877c051"). InnerVolumeSpecName "kube-api-access-8zglf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:03:41.420784 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.420724 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zglf\" (UniqueName: \"kubernetes.io/projected/3db85c56-fb5e-4ad1-98f7-5abb4877c051-kube-api-access-8zglf\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:41.420784 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.420781 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdkbp\" (UniqueName: \"kubernetes.io/projected/4c0e9405-2457-4dff-bffc-123dce2c6cae-kube-api-access-hdkbp\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:41.869610 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.869572 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c0e9405-2457-4dff-bffc-123dce2c6cae" containerID="10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0" exitCode=0 Jul 05 12:03:41.869816 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.869617 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" Jul 05 12:03:41.869816 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.869643 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" event={"ID":"4c0e9405-2457-4dff-bffc-123dce2c6cae","Type":"ContainerDied","Data":"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0"} Jul 05 12:03:41.869816 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.869679 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jdlgt" event={"ID":"4c0e9405-2457-4dff-bffc-123dce2c6cae","Type":"ContainerDied","Data":"4c8a77038cc50b97f3082212c5952e99ff798bf5c92b3ba5bb0a1135c2075ff4"} Jul 05 12:03:41.869816 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.869700 2574 scope.go:117] "RemoveContainer" containerID="10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0" Jul 05 12:03:41.871345 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.871266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c5568fd9-cn59h" event={"ID":"f024e22a-6f4d-49be-acf1-929da1510822","Type":"ContainerStarted","Data":"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8"} Jul 05 12:03:41.872561 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.872534 2574 generic.go:356] "Generic (PLEG): container finished" podID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" containerID="e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d" exitCode=0 Jul 05 12:03:41.872662 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.872564 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" Jul 05 12:03:41.872662 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.872616 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" event={"ID":"3db85c56-fb5e-4ad1-98f7-5abb4877c051","Type":"ContainerDied","Data":"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d"} Jul 05 12:03:41.872662 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.872651 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9cdd7bcc-hfpvz" event={"ID":"3db85c56-fb5e-4ad1-98f7-5abb4877c051","Type":"ContainerDied","Data":"5a225848c1134a28c1c6d3eb48c285df97c0a332d9236ca7f96b6f77429c799e"} Jul 05 12:03:41.879571 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.879552 2574 scope.go:117] "RemoveContainer" containerID="10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0" Jul 05 12:03:41.879861 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:41.879841 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0\": container with ID starting with 10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0 not found: ID does not exist" containerID="10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0" Jul 05 12:03:41.879959 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.879869 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0"} err="failed to get container status \"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0\": rpc error: code = NotFound desc = could not find container \"10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0\": container with ID starting with 10a9142ddc7b55037768da743da075b3276eae3fc71c816bd513d32a444a16d0 not found: ID does not exist" Jul 05 12:03:41.879959 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.879887 2574 scope.go:117] "RemoveContainer" containerID="e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d" Jul 05 12:03:41.887721 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.887704 2574 scope.go:117] "RemoveContainer" containerID="e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d" Jul 05 12:03:41.887998 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:41.887979 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d\": container with ID starting with e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d not found: ID does not exist" containerID="e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d" Jul 05 12:03:41.888080 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.888005 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d"} err="failed to get container status \"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d\": rpc error: code = NotFound desc = could not find container \"e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d\": container with ID starting with e91dbf5e9bb6e927c3529664581d71d43bd58677d53d27d16b8c922c0d7eff1d not found: ID does not exist" Jul 05 12:03:41.890164 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.890123 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69c5568fd9-cn59h" podStartSLOduration=1.332297335 podStartE2EDuration="1.890111486s" totalImagesPullingTime="557.814151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:41.888494365 +0000 UTC m=+606.146704826" watchObservedRunningTime="2026-07-05 12:03:41.890111486 +0000 UTC m=+606.148321946" Jul 05 12:03:41.903007 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.902981 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:41.907557 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.907529 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jdlgt"] Jul 05 12:03:41.912678 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.912655 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:41.912856 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.912837 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-fkc9m" podUID="1e138f58-6cd8-4741-85dc-088872c729e1" containerName="authorino" containerID="cri-o://52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071" gracePeriod=30 Jul 05 12:03:41.919682 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.919654 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:41.923971 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:41.923945 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c9cdd7bcc-hfpvz"] Jul 05 12:03:42.230854 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.230820 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:42.244375 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.244338 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" path="/var/lib/kubelet/pods/3db85c56-fb5e-4ad1-98f7-5abb4877c051/volumes" Jul 05 12:03:42.244664 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.244651 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c0e9405-2457-4dff-bffc-123dce2c6cae" path="/var/lib/kubelet/pods/4c0e9405-2457-4dff-bffc-123dce2c6cae/volumes" Jul 05 12:03:42.428678 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.428594 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfvrn\" (UniqueName: \"kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn\") pod \"1e138f58-6cd8-4741-85dc-088872c729e1\" (UID: \"1e138f58-6cd8-4741-85dc-088872c729e1\") " Jul 05 12:03:42.431297 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.431258 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn" (OuterVolumeSpecName: "kube-api-access-bfvrn") pod "1e138f58-6cd8-4741-85dc-088872c729e1" (UID: "1e138f58-6cd8-4741-85dc-088872c729e1"). InnerVolumeSpecName "kube-api-access-bfvrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:03:42.530280 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.530237 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfvrn\" (UniqueName: \"kubernetes.io/projected/1e138f58-6cd8-4741-85dc-088872c729e1-kube-api-access-bfvrn\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:03:42.880217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.880123 2574 generic.go:356] "Generic (PLEG): container finished" podID="1e138f58-6cd8-4741-85dc-088872c729e1" containerID="52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071" exitCode=0 Jul 05 12:03:42.880217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.880167 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fkc9m" Jul 05 12:03:42.880217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.880167 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkc9m" event={"ID":"1e138f58-6cd8-4741-85dc-088872c729e1","Type":"ContainerDied","Data":"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071"} Jul 05 12:03:42.880217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.880216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fkc9m" event={"ID":"1e138f58-6cd8-4741-85dc-088872c729e1","Type":"ContainerDied","Data":"e29c1320c3ecdedc6c330e7a438b863d03e55d347e5a72cdf168e2b1c27f85c5"} Jul 05 12:03:42.880523 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.880238 2574 scope.go:117] "RemoveContainer" containerID="52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071" Jul 05 12:03:42.888245 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.888053 2574 scope.go:117] "RemoveContainer" containerID="52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071" Jul 05 12:03:42.888478 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:42.888458 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071\": container with ID starting with 52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071 not found: ID does not exist" containerID="52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071" Jul 05 12:03:42.888557 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.888486 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071"} err="failed to get container status \"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071\": rpc error: code = NotFound desc = could not find container \"52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071\": container with ID starting with 52bd12b6cfecec595ef5b5c119d7d4ddb8c4cabf1fa9c65459f217505d5a7071 not found: ID does not exist" Jul 05 12:03:42.900559 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.900536 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:42.904193 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:42.904172 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fkc9m"] Jul 05 12:03:44.244184 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:44.244149 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e138f58-6cd8-4741-85dc-088872c729e1" path="/var/lib/kubelet/pods/1e138f58-6cd8-4741-85dc-088872c729e1/volumes" Jul 05 12:03:46.336626 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336592 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336906 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e138f58-6cd8-4741-85dc-088872c729e1" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336918 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e138f58-6cd8-4741-85dc-088872c729e1" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336926 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c0e9405-2457-4dff-bffc-123dce2c6cae" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336931 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c0e9405-2457-4dff-bffc-123dce2c6cae" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336945 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.336951 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.337006 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e138f58-6cd8-4741-85dc-088872c729e1" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.337012 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db85c56-fb5e-4ad1-98f7-5abb4877c051" containerName="authorino" Jul 05 12:03:46.337031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.337020 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c0e9405-2457-4dff-bffc-123dce2c6cae" containerName="authorino" Jul 05 12:03:46.348374 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.348347 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.349423 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.349396 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:03:46.351097 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.351065 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 05 12:03:46.351241 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.351101 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mnqf8\"" Jul 05 12:03:46.351241 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.351111 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 05 12:03:46.358620 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.358600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6lwx\" (UniqueName: \"kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.358716 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.358671 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.459948 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.459913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.460123 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.459962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6lwx\" (UniqueName: \"kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.460123 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:46.460044 2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 05 12:03:46.460123 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:03:46.460110 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert podName:aa903902-bb89-4276-ad6a-2aecd89140ed nodeName:}" failed. No retries permitted until 2026-07-05 12:03:46.960091371 +0000 UTC m=+611.218301810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert") pod "maas-controller-d4f9f4b57-4rnfh" (UID: "aa903902-bb89-4276-ad6a-2aecd89140ed") : secret "maas-controller-webhook-cert" not found Jul 05 12:03:46.470532 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.470504 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6lwx\" (UniqueName: \"kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.965191 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.965154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:46.967572 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:46.967540 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") pod \"maas-controller-d4f9f4b57-4rnfh\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:47.259130 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:47.259031 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:47.394474 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:47.394446 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:03:47.396794 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:03:47.396746 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa903902_bb89_4276_ad6a_2aecd89140ed.slice/crio-1e6e42ef31502c0b85399d946c1bf214f6926abb89e1f5ab777b21e19296c558 WatchSource:0}: Error finding container 1e6e42ef31502c0b85399d946c1bf214f6926abb89e1f5ab777b21e19296c558: Status 404 returned error can't find the container with id 1e6e42ef31502c0b85399d946c1bf214f6926abb89e1f5ab777b21e19296c558 Jul 05 12:03:47.906238 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:47.906200 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" event={"ID":"aa903902-bb89-4276-ad6a-2aecd89140ed","Type":"ContainerStarted","Data":"1e6e42ef31502c0b85399d946c1bf214f6926abb89e1f5ab777b21e19296c558"} Jul 05 12:03:49.915940 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:49.915899 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" event={"ID":"aa903902-bb89-4276-ad6a-2aecd89140ed","Type":"ContainerStarted","Data":"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501"} Jul 05 12:03:49.916376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:49.915979 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:03:49.931425 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:03:49.931369 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" podStartSLOduration=2.407946334 podStartE2EDuration="3.931352738s" totalImagesPullingTime="1.523406404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:03:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:03:49.930692472 +0000 UTC m=+614.188902950" watchObservedRunningTime="2026-07-05 12:03:49.931352738 +0000 UTC m=+614.189563198" Jul 05 12:04:00.926183 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:00.926108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:04:21.967712 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:21.967673 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-b4b8d7d5-jq6qn"] Jul 05 12:04:21.973504 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:21.973485 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:21.976180 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:21.976157 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 05 12:04:21.976300 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:21.976183 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-4cvrp\"" Jul 05 12:04:21.981024 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:21.980670 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b4b8d7d5-jq6qn"] Jul 05 12:04:22.075964 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.075925 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnnnf\" (UniqueName: \"kubernetes.io/projected/24c3a2b1-03a2-4d70-9244-f9b394082c2a-kube-api-access-hnnnf\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.076169 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.075992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3a2b1-03a2-4d70-9244-f9b394082c2a-openshift-service-ca\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.076169 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.076067 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.177596 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.177546 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnnnf\" (UniqueName: \"kubernetes.io/projected/24c3a2b1-03a2-4d70-9244-f9b394082c2a-kube-api-access-hnnnf\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.177776 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.177643 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3a2b1-03a2-4d70-9244-f9b394082c2a-openshift-service-ca\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.177776 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.177671 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.177854 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:04:22.177823 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 05 12:04:22.177979 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:04:22.177962 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls podName:24c3a2b1-03a2-4d70-9244-f9b394082c2a nodeName:}" failed. No retries permitted until 2026-07-05 12:04:22.677935433 +0000 UTC m=+646.936145876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls") pod "maas-api-b4b8d7d5-jq6qn" (UID: "24c3a2b1-03a2-4d70-9244-f9b394082c2a") : secret "maas-api-serving-cert" not found Jul 05 12:04:22.178359 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.178343 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c3a2b1-03a2-4d70-9244-f9b394082c2a-openshift-service-ca\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.186261 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.186234 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnnnf\" (UniqueName: \"kubernetes.io/projected/24c3a2b1-03a2-4d70-9244-f9b394082c2a-kube-api-access-hnnnf\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.681679 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.681639 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.684091 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.684068 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24c3a2b1-03a2-4d70-9244-f9b394082c2a-maas-api-tls\") pod \"maas-api-b4b8d7d5-jq6qn\" (UID: \"24c3a2b1-03a2-4d70-9244-f9b394082c2a\") " pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:22.884309 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:22.884273 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:23.018937 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:23.018910 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-b4b8d7d5-jq6qn"] Jul 05 12:04:23.021310 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:04:23.021280 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c3a2b1_03a2_4d70_9244_f9b394082c2a.slice/crio-406a25af092c38a874ead4c2eaa3f1f5d50ebd452157e8888727caac25375a43 WatchSource:0}: Error finding container 406a25af092c38a874ead4c2eaa3f1f5d50ebd452157e8888727caac25375a43: Status 404 returned error can't find the container with id 406a25af092c38a874ead4c2eaa3f1f5d50ebd452157e8888727caac25375a43 Jul 05 12:04:23.023000 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:23.022975 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 12:04:23.060755 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:23.060719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" event={"ID":"24c3a2b1-03a2-4d70-9244-f9b394082c2a","Type":"ContainerStarted","Data":"406a25af092c38a874ead4c2eaa3f1f5d50ebd452157e8888727caac25375a43"} Jul 05 12:04:25.071886 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:25.071851 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" event={"ID":"24c3a2b1-03a2-4d70-9244-f9b394082c2a","Type":"ContainerStarted","Data":"f917b9bb317514ea6faa303c060857a014d2d3e50b4cdb8a9921208a0de091c2"} Jul 05 12:04:25.072262 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:25.071992 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:04:25.088509 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:25.088452 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" podStartSLOduration=2.699613793 podStartE2EDuration="4.088435673s" totalImagesPullingTime="1.38882188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:04:25.08680164 +0000 UTC m=+649.345012125" watchObservedRunningTime="2026-07-05 12:04:25.088435673 +0000 UTC m=+649.346646133" Jul 05 12:04:31.081994 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:04:31.081961 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-b4b8d7d5-jq6qn" Jul 05 12:05:01.380461 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.380423 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-488dd"] Jul 05 12:05:01.383879 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.383848 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.386467 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.386438 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 05 12:05:01.387661 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.387632 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 05 12:05:01.387783 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.387661 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 05 12:05:01.387783 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.387666 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7msdg\"" Jul 05 12:05:01.392774 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.392733 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-488dd"] Jul 05 12:05:01.523274 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523236 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.523274 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wmmq\" (UniqueName: \"kubernetes.io/projected/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-kube-api-access-4wmmq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.523492 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.523492 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.523492 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523433 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.523492 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.523472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624476 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624428 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624476 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624481 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624722 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624537 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624722 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624560 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wmmq\" (UniqueName: \"kubernetes.io/projected/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-kube-api-access-4wmmq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624722 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624591 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.624722 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624613 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.625002 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.625049 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.624997 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.625084 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.625052 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.626708 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.626682 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.627034 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.627016 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.632750 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.632700 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wmmq\" (UniqueName: \"kubernetes.io/projected/4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4-kube-api-access-4wmmq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-488dd\" (UID: \"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.696547 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.696488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:01.827912 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:01.827886 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-488dd"] Jul 05 12:05:01.829719 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:05:01.829691 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd3d94d_0040_4dab_8b3b_bd8e909a4fe4.slice/crio-8b1b778948205e713c677d17525bda3161a33d8f5ac727a2d74874e6882b0908 WatchSource:0}: Error finding container 8b1b778948205e713c677d17525bda3161a33d8f5ac727a2d74874e6882b0908: Status 404 returned error can't find the container with id 8b1b778948205e713c677d17525bda3161a33d8f5ac727a2d74874e6882b0908 Jul 05 12:05:02.231928 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:02.231885 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" event={"ID":"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4","Type":"ContainerStarted","Data":"8b1b778948205e713c677d17525bda3161a33d8f5ac727a2d74874e6882b0908"} Jul 05 12:05:03.238269 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:03.238221 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" event={"ID":"4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4","Type":"ContainerStarted","Data":"f341237676fb094d67a833e9f5b32bbfdb7ce7a67213703e59eb56b18989c895"} Jul 05 12:05:03.254237 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:03.254185 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" podStartSLOduration=1.047869752 podStartE2EDuration="2.254157869s" totalImagesPullingTime="1.206288117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:05:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:05:03.253731053 +0000 UTC m=+687.511941537" watchObservedRunningTime="2026-07-05 12:05:03.254157869 +0000 UTC m=+687.512368395" Jul 05 12:05:11.697694 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:11.697649 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:11.698195 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:11.697732 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:11.710395 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:11.710371 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:12.286400 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:12.286370 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-488dd" Jul 05 12:05:34.885619 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.885582 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8"] Jul 05 12:05:34.889086 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.889065 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.892042 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.892022 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 05 12:05:34.897451 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.897426 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8"] Jul 05 12:05:34.912720 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.912849 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912741 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.912849 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.912945 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdec196-89c4-4567-9059-05c0623ac918-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.912945 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912900 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2cpg\" (UniqueName: \"kubernetes.io/projected/4cdec196-89c4-4567-9059-05c0623ac918-kube-api-access-w2cpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:34.913031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:34.912960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013599 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdec196-89c4-4567-9059-05c0623ac918-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013599 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013604 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2cpg\" (UniqueName: \"kubernetes.io/projected/4cdec196-89c4-4567-9059-05c0623ac918-kube-api-access-w2cpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013820 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013820 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013750 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013820 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013809 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.013942 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.013837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.014166 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.014141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.014249 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.014230 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.014296 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.014268 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.016069 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.016052 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cdec196-89c4-4567-9059-05c0623ac918-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.016341 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.016323 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cdec196-89c4-4567-9059-05c0623ac918-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.021314 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.021296 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2cpg\" (UniqueName: \"kubernetes.io/projected/4cdec196-89c4-4567-9059-05c0623ac918-kube-api-access-w2cpg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrwr8\" (UID: \"4cdec196-89c4-4567-9059-05c0623ac918\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.200442 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.200400 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:35.343780 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.343737 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8"] Jul 05 12:05:35.346039 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:05:35.346000 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdec196_89c4_4567_9059_05c0623ac918.slice/crio-9d9e3aaec946c6790cdbbd637567c2d1f3e27c4cf7c13f1895983959b447c9a4 WatchSource:0}: Error finding container 9d9e3aaec946c6790cdbbd637567c2d1f3e27c4cf7c13f1895983959b447c9a4: Status 404 returned error can't find the container with id 9d9e3aaec946c6790cdbbd637567c2d1f3e27c4cf7c13f1895983959b447c9a4 Jul 05 12:05:35.389072 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:35.389037 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" event={"ID":"4cdec196-89c4-4567-9059-05c0623ac918","Type":"ContainerStarted","Data":"9d9e3aaec946c6790cdbbd637567c2d1f3e27c4cf7c13f1895983959b447c9a4"} Jul 05 12:05:36.394449 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:36.394414 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" event={"ID":"4cdec196-89c4-4567-9059-05c0623ac918","Type":"ContainerStarted","Data":"f73b434eb3c27296c5a9c09ca756934689c62d3357bc351cfd2a0614bf700b97"} Jul 05 12:05:36.412602 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:36.412548 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" podStartSLOduration=2.242422293 podStartE2EDuration="2.412531101s" totalImagesPullingTime="170.108808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:05:36.410995858 +0000 UTC m=+720.669206335" watchObservedRunningTime="2026-07-05 12:05:36.412531101 +0000 UTC m=+720.670741561" Jul 05 12:05:45.095708 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.095674 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7"] Jul 05 12:05:45.098262 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.098239 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.100802 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.100777 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 05 12:05:45.107578 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.107526 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7"] Jul 05 12:05:45.200858 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.200817 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:45.201121 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.201100 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:45.213472 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.213447 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:45.215901 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.215874 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.216022 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.215905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87c34db-4e93-42de-b846-8fbb83149801-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.216022 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.215953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.216022 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.215991 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v65z\" (UniqueName: \"kubernetes.io/projected/f87c34db-4e93-42de-b846-8fbb83149801-kube-api-access-8v65z\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.216022 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.216013 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.216172 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.216031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.316927 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.316892 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v65z\" (UniqueName: \"kubernetes.io/projected/f87c34db-4e93-42de-b846-8fbb83149801-kube-api-access-8v65z\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317123 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.316953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317123 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.316984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317123 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317372 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317336 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87c34db-4e93-42de-b846-8fbb83149801-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317472 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317472 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317455 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317622 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317514 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.317807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.317749 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.319407 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.319384 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87c34db-4e93-42de-b846-8fbb83149801-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.319738 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.319720 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87c34db-4e93-42de-b846-8fbb83149801-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.324862 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.324841 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v65z\" (UniqueName: \"kubernetes.io/projected/f87c34db-4e93-42de-b846-8fbb83149801-kube-api-access-8v65z\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7\" (UID: \"f87c34db-4e93-42de-b846-8fbb83149801\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.408698 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.408613 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:45.449307 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.449275 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrwr8" Jul 05 12:05:45.544487 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:45.544464 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7"] Jul 05 12:05:45.547176 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:05:45.547144 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf87c34db_4e93_42de_b846_8fbb83149801.slice/crio-808ff8fc2bcd9bcfba9c2a232824393239e8bcf65340513eadf806c2ab59df3f WatchSource:0}: Error finding container 808ff8fc2bcd9bcfba9c2a232824393239e8bcf65340513eadf806c2ab59df3f: Status 404 returned error can't find the container with id 808ff8fc2bcd9bcfba9c2a232824393239e8bcf65340513eadf806c2ab59df3f Jul 05 12:05:46.443751 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:46.443713 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" event={"ID":"f87c34db-4e93-42de-b846-8fbb83149801","Type":"ContainerStarted","Data":"728871f62fbbe6f84ee29ac4559a40f456433650cfde242f7637a5a1675d0e8d"} Jul 05 12:05:46.443751 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:46.443756 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" event={"ID":"f87c34db-4e93-42de-b846-8fbb83149801","Type":"ContainerStarted","Data":"808ff8fc2bcd9bcfba9c2a232824393239e8bcf65340513eadf806c2ab59df3f"} Jul 05 12:05:46.460453 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:46.460401 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" podStartSLOduration=1.28993308 podStartE2EDuration="1.460384541s" totalImagesPullingTime="170.451461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:05:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:05:46.458585463 +0000 UTC m=+730.716795922" watchObservedRunningTime="2026-07-05 12:05:46.460384541 +0000 UTC m=+730.718595001" Jul 05 12:05:55.409288 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:55.409245 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:55.409288 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:55.409296 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:55.421727 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:55.421697 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:05:55.493904 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:05:55.493866 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7" Jul 05 12:07:04.048938 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.048851 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-674586996d-zfqgn"] Jul 05 12:07:04.051478 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.051459 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.059328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.059194 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674586996d-zfqgn"] Jul 05 12:07:04.122491 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.122451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-tls-cert\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.122491 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.122498 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfmv2\" (UniqueName: \"kubernetes.io/projected/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-kube-api-access-mfmv2\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.223463 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.223429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-tls-cert\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.223463 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.223464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfmv2\" (UniqueName: \"kubernetes.io/projected/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-kube-api-access-mfmv2\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.225947 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.225926 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-tls-cert\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.231743 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.231718 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfmv2\" (UniqueName: \"kubernetes.io/projected/30d0d8a5-2aed-44b1-a1e6-01aaa4d28939-kube-api-access-mfmv2\") pod \"authorino-674586996d-zfqgn\" (UID: \"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939\") " pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.361944 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.361859 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-674586996d-zfqgn" Jul 05 12:07:04.492974 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.492902 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-674586996d-zfqgn"] Jul 05 12:07:04.496449 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:07:04.496419 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30d0d8a5_2aed_44b1_a1e6_01aaa4d28939.slice/crio-4f69ced12fda3543b6739c7a55ac3fc3e473a946de9caef7065c5106e4644348 WatchSource:0}: Error finding container 4f69ced12fda3543b6739c7a55ac3fc3e473a946de9caef7065c5106e4644348: Status 404 returned error can't find the container with id 4f69ced12fda3543b6739c7a55ac3fc3e473a946de9caef7065c5106e4644348 Jul 05 12:07:04.807444 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:04.807407 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674586996d-zfqgn" event={"ID":"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939","Type":"ContainerStarted","Data":"4f69ced12fda3543b6739c7a55ac3fc3e473a946de9caef7065c5106e4644348"} Jul 05 12:07:05.814512 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:05.814464 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-674586996d-zfqgn" event={"ID":"30d0d8a5-2aed-44b1-a1e6-01aaa4d28939","Type":"ContainerStarted","Data":"9e800e6139e0bcfb0935b359e364cdd42aae5ccc02f3a520232236550767136a"} Jul 05 12:07:05.834064 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:05.833990 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-674586996d-zfqgn" podStartSLOduration=1.283764305 podStartE2EDuration="1.833968435s" totalImagesPullingTime="550.20413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:07:05.831563132 +0000 UTC m=+810.089773589" watchObservedRunningTime="2026-07-05 12:07:05.833968435 +0000 UTC m=+810.092178896" Jul 05 12:07:05.859103 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:05.859067 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:07:05.859380 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:05.859341 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-69c5568fd9-cn59h" podUID="f024e22a-6f4d-49be-acf1-929da1510822" containerName="authorino" containerID="cri-o://46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8" gracePeriod=30 Jul 05 12:07:06.115907 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.115879 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:07:06.139998 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.139963 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert\") pod \"f024e22a-6f4d-49be-acf1-929da1510822\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " Jul 05 12:07:06.140165 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.140069 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqjvr\" (UniqueName: \"kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr\") pod \"f024e22a-6f4d-49be-acf1-929da1510822\" (UID: \"f024e22a-6f4d-49be-acf1-929da1510822\") " Jul 05 12:07:06.142162 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.142127 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr" (OuterVolumeSpecName: "kube-api-access-fqjvr") pod "f024e22a-6f4d-49be-acf1-929da1510822" (UID: "f024e22a-6f4d-49be-acf1-929da1510822"). InnerVolumeSpecName "kube-api-access-fqjvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:07:06.152036 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.152004 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f024e22a-6f4d-49be-acf1-929da1510822" (UID: "f024e22a-6f4d-49be-acf1-929da1510822"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:07:06.241304 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.241259 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqjvr\" (UniqueName: \"kubernetes.io/projected/f024e22a-6f4d-49be-acf1-929da1510822-kube-api-access-fqjvr\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:07:06.241304 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.241311 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f024e22a-6f4d-49be-acf1-929da1510822-tls-cert\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:07:06.820022 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.819991 2574 generic.go:356] "Generic (PLEG): container finished" podID="f024e22a-6f4d-49be-acf1-929da1510822" containerID="46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8" exitCode=0 Jul 05 12:07:06.820437 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.820053 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69c5568fd9-cn59h" Jul 05 12:07:06.820437 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.820077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c5568fd9-cn59h" event={"ID":"f024e22a-6f4d-49be-acf1-929da1510822","Type":"ContainerDied","Data":"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8"} Jul 05 12:07:06.820437 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.820121 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69c5568fd9-cn59h" event={"ID":"f024e22a-6f4d-49be-acf1-929da1510822","Type":"ContainerDied","Data":"0e0c150e2284b79701a6c665ed49ac8587169a9567595ea30a74f0597c0e32d6"} Jul 05 12:07:06.820437 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.820141 2574 scope.go:117] "RemoveContainer" containerID="46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8" Jul 05 12:07:06.829121 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.829097 2574 scope.go:117] "RemoveContainer" containerID="46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8" Jul 05 12:07:06.829440 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:07:06.829414 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8\": container with ID starting with 46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8 not found: ID does not exist" containerID="46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8" Jul 05 12:07:06.829520 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.829453 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8"} err="failed to get container status \"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8\": rpc error: code = NotFound desc = could not find container \"46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8\": container with ID starting with 46afd1f1620b81b448bfbea370b9301f94d3349d5bb916b425c62e7a7801f7f8 not found: ID does not exist" Jul 05 12:07:06.840704 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.840675 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:07:06.845702 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:06.845679 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-69c5568fd9-cn59h"] Jul 05 12:07:08.243527 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:07:08.243496 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f024e22a-6f4d-49be-acf1-929da1510822" path="/var/lib/kubelet/pods/f024e22a-6f4d-49be-acf1-929da1510822/volumes" Jul 05 12:08:16.201470 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.201432 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:08:16.202047 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.201617 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" podUID="aa903902-bb89-4276-ad6a-2aecd89140ed" containerName="manager" containerID="cri-o://881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501" gracePeriod=30 Jul 05 12:08:16.450441 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.450413 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:08:16.572545 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.572465 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6lwx\" (UniqueName: \"kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx\") pod \"aa903902-bb89-4276-ad6a-2aecd89140ed\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " Jul 05 12:08:16.572693 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.572554 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") pod \"aa903902-bb89-4276-ad6a-2aecd89140ed\" (UID: \"aa903902-bb89-4276-ad6a-2aecd89140ed\") " Jul 05 12:08:16.574606 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.574572 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa903902-bb89-4276-ad6a-2aecd89140ed" (UID: "aa903902-bb89-4276-ad6a-2aecd89140ed"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:08:16.574727 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.574657 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx" (OuterVolumeSpecName: "kube-api-access-x6lwx") pod "aa903902-bb89-4276-ad6a-2aecd89140ed" (UID: "aa903902-bb89-4276-ad6a-2aecd89140ed"). InnerVolumeSpecName "kube-api-access-x6lwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:08:16.674098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.674063 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6lwx\" (UniqueName: \"kubernetes.io/projected/aa903902-bb89-4276-ad6a-2aecd89140ed-kube-api-access-x6lwx\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:08:16.674098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:16.674092 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa903902-bb89-4276-ad6a-2aecd89140ed-webhook-cert\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:08:17.131464 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.131424 2574 generic.go:356] "Generic (PLEG): container finished" podID="aa903902-bb89-4276-ad6a-2aecd89140ed" containerID="881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501" exitCode=0 Jul 05 12:08:17.131637 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.131473 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" Jul 05 12:08:17.131637 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.131511 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" event={"ID":"aa903902-bb89-4276-ad6a-2aecd89140ed","Type":"ContainerDied","Data":"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501"} Jul 05 12:08:17.131637 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.131568 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d4f9f4b57-4rnfh" event={"ID":"aa903902-bb89-4276-ad6a-2aecd89140ed","Type":"ContainerDied","Data":"1e6e42ef31502c0b85399d946c1bf214f6926abb89e1f5ab777b21e19296c558"} Jul 05 12:08:17.131637 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.131592 2574 scope.go:117] "RemoveContainer" containerID="881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501" Jul 05 12:08:17.140602 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.140583 2574 scope.go:117] "RemoveContainer" containerID="881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501" Jul 05 12:08:17.140923 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:08:17.140901 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501\": container with ID starting with 881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501 not found: ID does not exist" containerID="881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501" Jul 05 12:08:17.140999 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.140929 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501"} err="failed to get container status \"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501\": rpc error: code = NotFound desc = could not find container \"881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501\": container with ID starting with 881f3fb489eae60301e2012cc091987e5c616d669fb436c4601eb31575560501 not found: ID does not exist" Jul 05 12:08:17.151605 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.151578 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:08:17.154975 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:17.154951 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d4f9f4b57-4rnfh"] Jul 05 12:08:18.244239 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:08:18.244206 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa903902-bb89-4276-ad6a-2aecd89140ed" path="/var/lib/kubelet/pods/aa903902-bb89-4276-ad6a-2aecd89140ed/volumes" Jul 05 12:20:10.794827 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.794792 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:10.795330 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795311 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f024e22a-6f4d-49be-acf1-929da1510822" containerName="authorino" Jul 05 12:20:10.795376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795334 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f024e22a-6f4d-49be-acf1-929da1510822" containerName="authorino" Jul 05 12:20:10.795376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795349 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa903902-bb89-4276-ad6a-2aecd89140ed" containerName="manager" Jul 05 12:20:10.795376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795358 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa903902-bb89-4276-ad6a-2aecd89140ed" containerName="manager" Jul 05 12:20:10.795465 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795450 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f024e22a-6f4d-49be-acf1-929da1510822" containerName="authorino" Jul 05 12:20:10.795497 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.795466 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa903902-bb89-4276-ad6a-2aecd89140ed" containerName="manager" Jul 05 12:20:10.798554 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.798531 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.800962 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.800941 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-9b1d55bc-serving-cert\"" Jul 05 12:20:10.805945 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.805923 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:10.860295 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.860266 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.860414 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.860306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.860414 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.860335 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnbq\" (UniqueName: \"kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.961383 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.961355 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.961509 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.961395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.961509 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.961441 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnbq\" (UniqueName: \"kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.961509 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:10.961501 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-9b1d55bc-serving-cert: secret "maas-api-e2e-ait-9b1d55bc-serving-cert" not found Jul 05 12:20:10.961613 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:10.961577 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls podName:80a9ec69-3ea2-4db1-9f4d-7046d430943a nodeName:}" failed. No retries permitted until 2026-07-05 12:20:11.461557389 +0000 UTC m=+1595.719767832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls") pod "maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" (UID: "80a9ec69-3ea2-4db1-9f4d-7046d430943a") : secret "maas-api-e2e-ait-9b1d55bc-serving-cert" not found Jul 05 12:20:10.962053 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.962024 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:10.971942 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:10.971919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnbq\" (UniqueName: \"kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:11.465589 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:11.465558 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:11.468020 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:11.467992 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") pod \"maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:11.711425 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:11.711390 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:11.841399 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:11.841375 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:11.843604 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:20:11.843559 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a9ec69_3ea2_4db1_9f4d_7046d430943a.slice/crio-547fa006622fa5c0d93da5789c7d854be2d3e50ea0c55ef4d39a26ae2dbe718e WatchSource:0}: Error finding container 547fa006622fa5c0d93da5789c7d854be2d3e50ea0c55ef4d39a26ae2dbe718e: Status 404 returned error can't find the container with id 547fa006622fa5c0d93da5789c7d854be2d3e50ea0c55ef4d39a26ae2dbe718e Jul 05 12:20:11.844994 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:11.844955 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 12:20:12.309414 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:12.309385 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" event={"ID":"80a9ec69-3ea2-4db1-9f4d-7046d430943a","Type":"ContainerStarted","Data":"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac"} Jul 05 12:20:12.309524 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:12.309435 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" event={"ID":"80a9ec69-3ea2-4db1-9f4d-7046d430943a","Type":"ContainerStarted","Data":"547fa006622fa5c0d93da5789c7d854be2d3e50ea0c55ef4d39a26ae2dbe718e"} Jul 05 12:20:12.309524 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:12.309463 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:12.325943 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:12.325898 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" podStartSLOduration=1.930011023 podStartE2EDuration="2.325883644s" totalImagesPullingTime="395.872621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:20:12.324677729 +0000 UTC m=+1596.582888191" watchObservedRunningTime="2026-07-05 12:20:12.325883644 +0000 UTC m=+1596.584094104" Jul 05 12:20:16.249326 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.249294 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:16.249878 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.249487 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" podUID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" containerName="maas-api" containerID="cri-o://f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac" gracePeriod=30 Jul 05 12:20:16.491345 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.491262 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:16.607947 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.607875 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") pod \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " Jul 05 12:20:16.607947 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.607938 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca\") pod \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " Jul 05 12:20:16.608112 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.607964 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cnbq\" (UniqueName: \"kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq\") pod \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\" (UID: \"80a9ec69-3ea2-4db1-9f4d-7046d430943a\") " Jul 05 12:20:16.608368 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.608343 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80a9ec69-3ea2-4db1-9f4d-7046d430943a" (UID: "80a9ec69-3ea2-4db1-9f4d-7046d430943a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:20:16.610008 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.609986 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80a9ec69-3ea2-4db1-9f4d-7046d430943a" (UID: "80a9ec69-3ea2-4db1-9f4d-7046d430943a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:20:16.610096 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.610081 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq" (OuterVolumeSpecName: "kube-api-access-4cnbq") pod "80a9ec69-3ea2-4db1-9f4d-7046d430943a" (UID: "80a9ec69-3ea2-4db1-9f4d-7046d430943a"). InnerVolumeSpecName "kube-api-access-4cnbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:20:16.708928 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.708898 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a9ec69-3ea2-4db1-9f4d-7046d430943a-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:16.708928 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.708921 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cnbq\" (UniqueName: \"kubernetes.io/projected/80a9ec69-3ea2-4db1-9f4d-7046d430943a-kube-api-access-4cnbq\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:16.709069 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:16.708935 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a9ec69-3ea2-4db1-9f4d-7046d430943a-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:17.332717 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.332681 2574 generic.go:356] "Generic (PLEG): container finished" podID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" containerID="f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac" exitCode=0 Jul 05 12:20:17.333141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.332726 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" Jul 05 12:20:17.333141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.332752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" event={"ID":"80a9ec69-3ea2-4db1-9f4d-7046d430943a","Type":"ContainerDied","Data":"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac"} Jul 05 12:20:17.333141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.332814 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b" event={"ID":"80a9ec69-3ea2-4db1-9f4d-7046d430943a","Type":"ContainerDied","Data":"547fa006622fa5c0d93da5789c7d854be2d3e50ea0c55ef4d39a26ae2dbe718e"} Jul 05 12:20:17.333141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.332837 2574 scope.go:117] "RemoveContainer" containerID="f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac" Jul 05 12:20:17.341954 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.341931 2574 scope.go:117] "RemoveContainer" containerID="f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac" Jul 05 12:20:17.342236 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:17.342212 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac\": container with ID starting with f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac not found: ID does not exist" containerID="f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac" Jul 05 12:20:17.342316 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.342246 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac"} err="failed to get container status \"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac\": rpc error: code = NotFound desc = could not find container \"f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac\": container with ID starting with f0abefcebcfa229f49282d2cf62039c09f5cd56f6e3f7b8a07dd02a6b5e755ac not found: ID does not exist" Jul 05 12:20:17.354209 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.354185 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:17.357941 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:17.357921 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-9b1d55bc-77769854d5-6jd7b"] Jul 05 12:20:18.244032 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:18.243995 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" path="/var/lib/kubelet/pods/80a9ec69-3ea2-4db1-9f4d-7046d430943a/volumes" Jul 05 12:20:23.282841 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.282800 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:23.283302 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.283262 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" containerName="maas-api" Jul 05 12:20:23.283302 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.283281 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" containerName="maas-api" Jul 05 12:20:23.283416 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.283399 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a9ec69-3ea2-4db1-9f4d-7046d430943a" containerName="maas-api" Jul 05 12:20:23.287678 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.287656 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.290165 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.290142 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-c2af4718-serving-cert\"" Jul 05 12:20:23.295866 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.295843 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:23.468652 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.468615 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.468652 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.468655 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84p6z\" (UniqueName: \"kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.468886 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.468685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.570061 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.569979 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.570188 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.570083 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.570188 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.570109 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84p6z\" (UniqueName: \"kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.570797 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.570780 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.572296 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.572278 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.577514 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.577493 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84p6z\" (UniqueName: \"kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z\") pod \"maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.598848 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.598824 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:23.724400 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:23.724309 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:23.726582 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:20:23.726554 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa266be4_52ad_4cc5_a970_39fd5039b48d.slice/crio-567c857507a95833a7e4c3fb11acedd17733f10514f1e759e0b8b8638933db20 WatchSource:0}: Error finding container 567c857507a95833a7e4c3fb11acedd17733f10514f1e759e0b8b8638933db20: Status 404 returned error can't find the container with id 567c857507a95833a7e4c3fb11acedd17733f10514f1e759e0b8b8638933db20 Jul 05 12:20:24.364256 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:24.364158 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" event={"ID":"fa266be4-52ad-4cc5-a970-39fd5039b48d","Type":"ContainerStarted","Data":"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9"} Jul 05 12:20:24.364256 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:24.364232 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:24.364647 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:24.364259 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" event={"ID":"fa266be4-52ad-4cc5-a970-39fd5039b48d","Type":"ContainerStarted","Data":"567c857507a95833a7e4c3fb11acedd17733f10514f1e759e0b8b8638933db20"} Jul 05 12:20:24.384907 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:24.384851 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" podStartSLOduration=1.059469941 podStartE2EDuration="1.384836039s" totalImagesPullingTime="325.366098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:20:24.383289573 +0000 UTC m=+1608.641500033" watchObservedRunningTime="2026-07-05 12:20:24.384836039 +0000 UTC m=+1608.643046499" Jul 05 12:20:28.670147 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:28.670055 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:28.670592 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:28.670277 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" podUID="fa266be4-52ad-4cc5-a970-39fd5039b48d" containerName="maas-api" containerID="cri-o://6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9" gracePeriod=30 Jul 05 12:20:28.914701 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:28.914678 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:29.012698 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.012674 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84p6z\" (UniqueName: \"kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z\") pod \"fa266be4-52ad-4cc5-a970-39fd5039b48d\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " Jul 05 12:20:29.012853 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.012728 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls\") pod \"fa266be4-52ad-4cc5-a970-39fd5039b48d\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " Jul 05 12:20:29.012853 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.012749 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca\") pod \"fa266be4-52ad-4cc5-a970-39fd5039b48d\" (UID: \"fa266be4-52ad-4cc5-a970-39fd5039b48d\") " Jul 05 12:20:29.013140 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.013118 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa266be4-52ad-4cc5-a970-39fd5039b48d" (UID: "fa266be4-52ad-4cc5-a970-39fd5039b48d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:20:29.014690 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.014666 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z" (OuterVolumeSpecName: "kube-api-access-84p6z") pod "fa266be4-52ad-4cc5-a970-39fd5039b48d" (UID: "fa266be4-52ad-4cc5-a970-39fd5039b48d"). InnerVolumeSpecName "kube-api-access-84p6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:20:29.014869 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.014695 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa266be4-52ad-4cc5-a970-39fd5039b48d" (UID: "fa266be4-52ad-4cc5-a970-39fd5039b48d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:20:29.113679 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.113651 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84p6z\" (UniqueName: \"kubernetes.io/projected/fa266be4-52ad-4cc5-a970-39fd5039b48d-kube-api-access-84p6z\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:29.113679 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.113674 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa266be4-52ad-4cc5-a970-39fd5039b48d-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:29.113908 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.113684 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa266be4-52ad-4cc5-a970-39fd5039b48d-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:29.387438 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.387363 2574 generic.go:356] "Generic (PLEG): container finished" podID="fa266be4-52ad-4cc5-a970-39fd5039b48d" containerID="6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9" exitCode=0 Jul 05 12:20:29.387438 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.387412 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" Jul 05 12:20:29.387595 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.387449 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" event={"ID":"fa266be4-52ad-4cc5-a970-39fd5039b48d","Type":"ContainerDied","Data":"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9"} Jul 05 12:20:29.387595 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.387505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl" event={"ID":"fa266be4-52ad-4cc5-a970-39fd5039b48d","Type":"ContainerDied","Data":"567c857507a95833a7e4c3fb11acedd17733f10514f1e759e0b8b8638933db20"} Jul 05 12:20:29.387595 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.387533 2574 scope.go:117] "RemoveContainer" containerID="6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9" Jul 05 12:20:29.399051 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.399034 2574 scope.go:117] "RemoveContainer" containerID="6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9" Jul 05 12:20:29.399296 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:29.399278 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9\": container with ID starting with 6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9 not found: ID does not exist" containerID="6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9" Jul 05 12:20:29.399334 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.399308 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9"} err="failed to get container status \"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9\": rpc error: code = NotFound desc = could not find container \"6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9\": container with ID starting with 6bcb7e53e601302213c6ccdb199ff7ef2e2ba661c707a537dc86b2a8c96c2dc9 not found: ID does not exist" Jul 05 12:20:29.409117 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.409094 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:29.414129 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:29.414108 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-c2af4718-59b6f8f998-9g2jl"] Jul 05 12:20:30.243720 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:30.243685 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa266be4-52ad-4cc5-a970-39fd5039b48d" path="/var/lib/kubelet/pods/fa266be4-52ad-4cc5-a970-39fd5039b48d/volumes" Jul 05 12:20:42.067371 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.067339 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:42.067806 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.067789 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa266be4-52ad-4cc5-a970-39fd5039b48d" containerName="maas-api" Jul 05 12:20:42.067868 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.067810 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa266be4-52ad-4cc5-a970-39fd5039b48d" containerName="maas-api" Jul 05 12:20:42.067905 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.067894 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa266be4-52ad-4cc5-a970-39fd5039b48d" containerName="maas-api" Jul 05 12:20:42.071056 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.071037 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.073512 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.073489 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-d0668d61-serving-cert\"" Jul 05 12:20:42.091217 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.090006 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:42.115207 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.115180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrn8\" (UniqueName: \"kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.115354 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.115215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.115354 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.115246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.216841 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.216804 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.216983 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.216897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.217049 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.217018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrn8\" (UniqueName: \"kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.217124 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:42.217088 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-d0668d61-serving-cert: secret "maas-api-e2e-derive-d0668d61-serving-cert" not found Jul 05 12:20:42.217205 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:42.217186 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls podName:99396732-98d5-46b9-b521-2fcf8fc731af nodeName:}" failed. No retries permitted until 2026-07-05 12:20:42.717161106 +0000 UTC m=+1626.975371555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls") pod "maas-api-e2e-derive-d0668d61-5d595f9784-b5962" (UID: "99396732-98d5-46b9-b521-2fcf8fc731af") : secret "maas-api-e2e-derive-d0668d61-serving-cert" not found Jul 05 12:20:42.217581 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.217558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.226136 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.226111 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrn8\" (UniqueName: \"kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.721320 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.721288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.723592 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.723560 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") pod \"maas-api-e2e-derive-d0668d61-5d595f9784-b5962\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:42.984173 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:42.984101 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:43.108958 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:43.108882 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:43.111399 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:20:43.111369 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99396732_98d5_46b9_b521_2fcf8fc731af.slice/crio-553caac82e94a1b17b1d2c1bed78ef8d7120ca5fe964b42e4fc520f779cce87f WatchSource:0}: Error finding container 553caac82e94a1b17b1d2c1bed78ef8d7120ca5fe964b42e4fc520f779cce87f: Status 404 returned error can't find the container with id 553caac82e94a1b17b1d2c1bed78ef8d7120ca5fe964b42e4fc520f779cce87f Jul 05 12:20:43.448565 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:43.448524 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" event={"ID":"99396732-98d5-46b9-b521-2fcf8fc731af","Type":"ContainerStarted","Data":"553caac82e94a1b17b1d2c1bed78ef8d7120ca5fe964b42e4fc520f779cce87f"} Jul 05 12:20:44.454392 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:44.454354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" event={"ID":"99396732-98d5-46b9-b521-2fcf8fc731af","Type":"ContainerStarted","Data":"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4"} Jul 05 12:20:44.454833 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:44.454417 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:44.471501 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:44.471455 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" podStartSLOduration=2.125960502 podStartE2EDuration="2.471441165s" totalImagesPullingTime="345.480663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:20:44.470186056 +0000 UTC m=+1628.728396516" watchObservedRunningTime="2026-07-05 12:20:44.471441165 +0000 UTC m=+1628.729651622" Jul 05 12:20:46.837125 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:46.837091 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:46.837592 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:46.837280 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" podUID="99396732-98d5-46b9-b521-2fcf8fc731af" containerName="maas-api" containerID="cri-o://7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4" gracePeriod=30 Jul 05 12:20:47.081627 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.081594 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:47.162820 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.162736 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") pod \"99396732-98d5-46b9-b521-2fcf8fc731af\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " Jul 05 12:20:47.162820 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.162823 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca\") pod \"99396732-98d5-46b9-b521-2fcf8fc731af\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " Jul 05 12:20:47.163035 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.162856 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vrn8\" (UniqueName: \"kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8\") pod \"99396732-98d5-46b9-b521-2fcf8fc731af\" (UID: \"99396732-98d5-46b9-b521-2fcf8fc731af\") " Jul 05 12:20:47.163232 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.163205 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99396732-98d5-46b9-b521-2fcf8fc731af" (UID: "99396732-98d5-46b9-b521-2fcf8fc731af"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:20:47.164895 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.164865 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8" (OuterVolumeSpecName: "kube-api-access-2vrn8") pod "99396732-98d5-46b9-b521-2fcf8fc731af" (UID: "99396732-98d5-46b9-b521-2fcf8fc731af"). InnerVolumeSpecName "kube-api-access-2vrn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:20:47.164996 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.164893 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99396732-98d5-46b9-b521-2fcf8fc731af" (UID: "99396732-98d5-46b9-b521-2fcf8fc731af"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:20:47.263655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.263621 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99396732-98d5-46b9-b521-2fcf8fc731af-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:47.263655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.263651 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99396732-98d5-46b9-b521-2fcf8fc731af-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:47.263655 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.263661 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vrn8\" (UniqueName: \"kubernetes.io/projected/99396732-98d5-46b9-b521-2fcf8fc731af-kube-api-access-2vrn8\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:20:47.472355 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.472318 2574 generic.go:356] "Generic (PLEG): container finished" podID="99396732-98d5-46b9-b521-2fcf8fc731af" containerID="7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4" exitCode=0 Jul 05 12:20:47.472555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.472362 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" Jul 05 12:20:47.472555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.472381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" event={"ID":"99396732-98d5-46b9-b521-2fcf8fc731af","Type":"ContainerDied","Data":"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4"} Jul 05 12:20:47.472555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.472432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962" event={"ID":"99396732-98d5-46b9-b521-2fcf8fc731af","Type":"ContainerDied","Data":"553caac82e94a1b17b1d2c1bed78ef8d7120ca5fe964b42e4fc520f779cce87f"} Jul 05 12:20:47.472555 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.472462 2574 scope.go:117] "RemoveContainer" containerID="7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4" Jul 05 12:20:47.482384 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.482362 2574 scope.go:117] "RemoveContainer" containerID="7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4" Jul 05 12:20:47.482639 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:20:47.482619 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4\": container with ID starting with 7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4 not found: ID does not exist" containerID="7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4" Jul 05 12:20:47.482710 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.482652 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4"} err="failed to get container status \"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4\": rpc error: code = NotFound desc = could not find container \"7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4\": container with ID starting with 7384196cb319a2f2f8490dc1bbe3d37290ed1479d14addbd2a911cfe72e741f4 not found: ID does not exist" Jul 05 12:20:47.493787 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.493746 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:47.497376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:47.497353 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-d0668d61-5d595f9784-b5962"] Jul 05 12:20:48.244661 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:20:48.244619 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99396732-98d5-46b9-b521-2fcf8fc731af" path="/var/lib/kubelet/pods/99396732-98d5-46b9-b521-2fcf8fc731af/volumes" Jul 05 12:22:29.871432 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.871355 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:29.871904 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.871868 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99396732-98d5-46b9-b521-2fcf8fc731af" containerName="maas-api" Jul 05 12:22:29.871904 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.871886 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99396732-98d5-46b9-b521-2fcf8fc731af" containerName="maas-api" Jul 05 12:22:29.872042 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.871993 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99396732-98d5-46b9-b521-2fcf8fc731af" containerName="maas-api" Jul 05 12:22:29.875059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.875041 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:29.877642 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.877623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-e2215ed8-serving-cert\"" Jul 05 12:22:29.885394 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.885369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:29.946752 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.946722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:29.946894 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.946778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:29.946894 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:29.946881 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mksq\" (UniqueName: \"kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.048136 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.048109 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.048247 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.048139 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.048247 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.048178 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mksq\" (UniqueName: \"kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.048358 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:22:30.048263 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-e2215ed8-serving-cert: secret "maas-api-e2e-mt-e2215ed8-serving-cert" not found Jul 05 12:22:30.048358 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:22:30.048335 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls podName:0cdba8f1-9882-4079-8833-a278a9469802 nodeName:}" failed. No retries permitted until 2026-07-05 12:22:30.548313099 +0000 UTC m=+1734.806523543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls") pod "maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" (UID: "0cdba8f1-9882-4079-8833-a278a9469802") : secret "maas-api-e2e-mt-e2215ed8-serving-cert" not found Jul 05 12:22:30.048756 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.048737 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.057065 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.057038 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mksq\" (UniqueName: \"kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.552216 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.552172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.554569 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.554538 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") pod \"maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.785181 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.785149 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:30.910596 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:30.910572 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:30.913197 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:22:30.913171 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cdba8f1_9882_4079_8833_a278a9469802.slice/crio-7f10e0eeb18ecbe658a2202d65262d0e1329f89388863f198aaa225b0e0397fb WatchSource:0}: Error finding container 7f10e0eeb18ecbe658a2202d65262d0e1329f89388863f198aaa225b0e0397fb: Status 404 returned error can't find the container with id 7f10e0eeb18ecbe658a2202d65262d0e1329f89388863f198aaa225b0e0397fb Jul 05 12:22:31.912800 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:31.912745 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" event={"ID":"0cdba8f1-9882-4079-8833-a278a9469802","Type":"ContainerStarted","Data":"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54"} Jul 05 12:22:31.913181 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:31.912818 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" event={"ID":"0cdba8f1-9882-4079-8833-a278a9469802","Type":"ContainerStarted","Data":"7f10e0eeb18ecbe658a2202d65262d0e1329f89388863f198aaa225b0e0397fb"} Jul 05 12:22:31.913181 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:31.912846 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:31.929825 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:31.929779 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" podStartSLOduration=2.440631148 podStartE2EDuration="2.929748804s" totalImagesPullingTime="489.117656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:22:31.928124335 +0000 UTC m=+1736.186334797" watchObservedRunningTime="2026-07-05 12:22:31.929748804 +0000 UTC m=+1736.187959267" Jul 05 12:22:37.922802 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:37.922771 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:41.857560 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:41.857527 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:41.858089 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:41.857714 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" podUID="0cdba8f1-9882-4079-8833-a278a9469802" containerName="maas-api" containerID="cri-o://eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54" gracePeriod=30 Jul 05 12:22:42.091333 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.091308 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:42.250738 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.250712 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca\") pod \"0cdba8f1-9882-4079-8833-a278a9469802\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " Jul 05 12:22:42.250880 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.250874 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") pod \"0cdba8f1-9882-4079-8833-a278a9469802\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " Jul 05 12:22:42.250956 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.250891 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mksq\" (UniqueName: \"kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq\") pod \"0cdba8f1-9882-4079-8833-a278a9469802\" (UID: \"0cdba8f1-9882-4079-8833-a278a9469802\") " Jul 05 12:22:42.251036 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.251010 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cdba8f1-9882-4079-8833-a278a9469802" (UID: "0cdba8f1-9882-4079-8833-a278a9469802"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:22:42.252884 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.252858 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cdba8f1-9882-4079-8833-a278a9469802" (UID: "0cdba8f1-9882-4079-8833-a278a9469802"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:22:42.253004 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.252967 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq" (OuterVolumeSpecName: "kube-api-access-8mksq") pod "0cdba8f1-9882-4079-8833-a278a9469802" (UID: "0cdba8f1-9882-4079-8833-a278a9469802"). InnerVolumeSpecName "kube-api-access-8mksq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:22:42.351654 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.351625 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cdba8f1-9882-4079-8833-a278a9469802-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:22:42.351654 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.351651 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mksq\" (UniqueName: \"kubernetes.io/projected/0cdba8f1-9882-4079-8833-a278a9469802-kube-api-access-8mksq\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:22:42.351840 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.351665 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cdba8f1-9882-4079-8833-a278a9469802-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:22:42.969373 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.969339 2574 generic.go:356] "Generic (PLEG): container finished" podID="0cdba8f1-9882-4079-8833-a278a9469802" containerID="eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54" exitCode=0 Jul 05 12:22:42.969807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.969380 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" Jul 05 12:22:42.969807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.969389 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" event={"ID":"0cdba8f1-9882-4079-8833-a278a9469802","Type":"ContainerDied","Data":"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54"} Jul 05 12:22:42.969807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.969437 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9" event={"ID":"0cdba8f1-9882-4079-8833-a278a9469802","Type":"ContainerDied","Data":"7f10e0eeb18ecbe658a2202d65262d0e1329f89388863f198aaa225b0e0397fb"} Jul 05 12:22:42.969807 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.969463 2574 scope.go:117] "RemoveContainer" containerID="eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54" Jul 05 12:22:42.978358 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.978341 2574 scope.go:117] "RemoveContainer" containerID="eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54" Jul 05 12:22:42.978579 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:22:42.978563 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54\": container with ID starting with eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54 not found: ID does not exist" containerID="eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54" Jul 05 12:22:42.978630 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.978586 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54"} err="failed to get container status \"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54\": rpc error: code = NotFound desc = could not find container \"eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54\": container with ID starting with eff70a017a2d3e91f50b05492746a00989a74f5c4b4a98fad8d9d9568196fc54 not found: ID does not exist" Jul 05 12:22:42.989771 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.989739 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:42.998113 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:42.995226 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-e2215ed8-77c767c557-wqsd9"] Jul 05 12:22:44.244973 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:22:44.244936 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cdba8f1-9882-4079-8833-a278a9469802" path="/var/lib/kubelet/pods/0cdba8f1-9882-4079-8833-a278a9469802/volumes" Jul 05 12:23:09.921896 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.921864 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:09.922416 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.922259 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cdba8f1-9882-4079-8833-a278a9469802" containerName="maas-api" Jul 05 12:23:09.922416 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.922277 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cdba8f1-9882-4079-8833-a278a9469802" containerName="maas-api" Jul 05 12:23:09.922416 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.922366 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cdba8f1-9882-4079-8833-a278a9469802" containerName="maas-api" Jul 05 12:23:09.925387 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.925364 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:09.927899 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.927877 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-c31da5f5-serving-cert\"" Jul 05 12:23:09.936891 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:09.936866 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:10.007163 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.007115 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.007343 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.007177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.007343 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.007240 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txl2n\" (UniqueName: \"kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.107805 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.107738 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.108009 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.107836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.108009 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.107866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txl2n\" (UniqueName: \"kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.108009 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:23:10.107993 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-c31da5f5-serving-cert: secret "maas-api-e2e-mt-c31da5f5-serving-cert" not found Jul 05 12:23:10.108186 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:23:10.108059 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls podName:8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9 nodeName:}" failed. No retries permitted until 2026-07-05 12:23:10.608041532 +0000 UTC m=+1774.866251970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls") pod "maas-api-e2e-mt-c31da5f5-5557675697-sf75q" (UID: "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9") : secret "maas-api-e2e-mt-c31da5f5-serving-cert" not found Jul 05 12:23:10.108429 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.108409 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.117942 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.117920 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txl2n\" (UniqueName: \"kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.612616 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.612567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.615043 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.615014 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") pod \"maas-api-e2e-mt-c31da5f5-5557675697-sf75q\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.837752 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.837707 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:10.970072 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:10.970040 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:10.971865 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:23:10.971837 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0b973b_2d75_4d5b_99dc_2f9f0604c4e9.slice/crio-7e249c9485c52243c04f6bd43f90a78e52344abe30b011e724c97f2df24b1720 WatchSource:0}: Error finding container 7e249c9485c52243c04f6bd43f90a78e52344abe30b011e724c97f2df24b1720: Status 404 returned error can't find the container with id 7e249c9485c52243c04f6bd43f90a78e52344abe30b011e724c97f2df24b1720 Jul 05 12:23:11.097602 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:11.097565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" event={"ID":"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9","Type":"ContainerStarted","Data":"7e249c9485c52243c04f6bd43f90a78e52344abe30b011e724c97f2df24b1720"} Jul 05 12:23:12.103624 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:12.103583 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" event={"ID":"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9","Type":"ContainerStarted","Data":"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d"} Jul 05 12:23:12.104092 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:12.103722 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:12.120253 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:12.120200 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" podStartSLOduration=2.647250263 podStartE2EDuration="3.120180866s" totalImagesPullingTime="472.930603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:23:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:23:12.118434916 +0000 UTC m=+1776.376645378" watchObservedRunningTime="2026-07-05 12:23:12.120180866 +0000 UTC m=+1776.378391325" Jul 05 12:23:18.114676 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:18.114640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:42.521898 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.521861 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:42.522496 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.522252 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" podUID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" containerName="maas-api" containerID="cri-o://30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d" gracePeriod=30 Jul 05 12:23:42.762823 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.762798 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:42.818251 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.818158 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txl2n\" (UniqueName: \"kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n\") pod \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " Jul 05 12:23:42.818251 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.818233 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") pod \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " Jul 05 12:23:42.818485 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.818279 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca\") pod \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\" (UID: \"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9\") " Jul 05 12:23:42.818663 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.818636 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" (UID: "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:23:42.820270 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.820248 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" (UID: "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:23:42.820373 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.820313 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n" (OuterVolumeSpecName: "kube-api-access-txl2n") pod "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" (UID: "8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9"). InnerVolumeSpecName "kube-api-access-txl2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:23:42.919196 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.919162 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:23:42.919196 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.919190 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:23:42.919196 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:42.919199 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txl2n\" (UniqueName: \"kubernetes.io/projected/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9-kube-api-access-txl2n\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:23:43.246320 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.246285 2574 generic.go:356] "Generic (PLEG): container finished" podID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" containerID="30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d" exitCode=0 Jul 05 12:23:43.246496 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.246335 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" Jul 05 12:23:43.246496 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.246371 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" event={"ID":"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9","Type":"ContainerDied","Data":"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d"} Jul 05 12:23:43.246496 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.246429 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q" event={"ID":"8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9","Type":"ContainerDied","Data":"7e249c9485c52243c04f6bd43f90a78e52344abe30b011e724c97f2df24b1720"} Jul 05 12:23:43.246496 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.246458 2574 scope.go:117] "RemoveContainer" containerID="30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d" Jul 05 12:23:43.264006 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.263981 2574 scope.go:117] "RemoveContainer" containerID="30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d" Jul 05 12:23:43.264272 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:23:43.264255 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d\": container with ID starting with 30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d not found: ID does not exist" containerID="30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d" Jul 05 12:23:43.264316 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.264279 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d"} err="failed to get container status \"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d\": rpc error: code = NotFound desc = could not find container \"30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d\": container with ID starting with 30dee88e9be2b4bf0bbb5ca863cd82ddae455cabd00fbfe0c9f94d2780b8e55d not found: ID does not exist" Jul 05 12:23:43.272671 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.272641 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:43.274622 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:43.274600 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-c31da5f5-5557675697-sf75q"] Jul 05 12:23:44.243832 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:23:44.243797 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" path="/var/lib/kubelet/pods/8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9/volumes" Jul 05 12:25:32.886071 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.886038 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:25:32.891098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.886406 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" containerName="maas-api" Jul 05 12:25:32.891098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.886417 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" containerName="maas-api" Jul 05 12:25:32.891098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.886467 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0b973b-2d75-4d5b-99dc-2f9f0604c4e9" containerName="maas-api" Jul 05 12:25:32.892326 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.892304 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:32.895392 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.895369 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-1d2467-serving-cert\"" Jul 05 12:25:32.901873 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.901849 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:25:32.992094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.992057 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:32.992255 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.992195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jw4g\" (UniqueName: \"kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:32.992255 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:32.992233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.093455 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.093419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jw4g\" (UniqueName: \"kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.093455 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.093458 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.093668 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.093480 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.093668 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:33.093572 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-1d2467-serving-cert: secret "maas-api-e2e-inf-a-1d2467-serving-cert" not found Jul 05 12:25:33.093668 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:33.093637 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls podName:4c26e211-6b4e-46e5-ad9b-e89ce11dd369 nodeName:}" failed. No retries permitted until 2026-07-05 12:25:33.593621786 +0000 UTC m=+1917.851832229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls") pod "maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" (UID: "4c26e211-6b4e-46e5-ad9b-e89ce11dd369") : secret "maas-api-e2e-inf-a-1d2467-serving-cert" not found Jul 05 12:25:33.094116 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.094100 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.104854 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.104833 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jw4g\" (UniqueName: \"kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.597145 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:33.597106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:33.597322 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:33.597260 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-1d2467-serving-cert: secret "maas-api-e2e-inf-a-1d2467-serving-cert" not found Jul 05 12:25:33.597365 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:33.597333 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls podName:4c26e211-6b4e-46e5-ad9b-e89ce11dd369 nodeName:}" failed. No retries permitted until 2026-07-05 12:25:34.597314943 +0000 UTC m=+1918.855525392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls") pod "maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" (UID: "4c26e211-6b4e-46e5-ad9b-e89ce11dd369") : secret "maas-api-e2e-inf-a-1d2467-serving-cert" not found Jul 05 12:25:34.608150 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:34.608110 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:34.610683 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:34.610658 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:34.704385 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:34.704349 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:34.829781 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:34.829728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:25:34.831870 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:25:34.831840 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c26e211_6b4e_46e5_ad9b_e89ce11dd369.slice/crio-a369015444e0e295ce9fbb63b73ae3fd3ffe3f1f370c56ec70d6f253656368c9 WatchSource:0}: Error finding container a369015444e0e295ce9fbb63b73ae3fd3ffe3f1f370c56ec70d6f253656368c9: Status 404 returned error can't find the container with id a369015444e0e295ce9fbb63b73ae3fd3ffe3f1f370c56ec70d6f253656368c9 Jul 05 12:25:34.833299 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:34.833279 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 05 12:25:35.750209 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:35.750172 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" event={"ID":"4c26e211-6b4e-46e5-ad9b-e89ce11dd369","Type":"ContainerStarted","Data":"22015d5699e2d075249719f8db4c5adc4651816f0e9308a86bf0012d6d3a2938"} Jul 05 12:25:35.750587 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:35.750225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" event={"ID":"4c26e211-6b4e-46e5-ad9b-e89ce11dd369","Type":"ContainerStarted","Data":"a369015444e0e295ce9fbb63b73ae3fd3ffe3f1f370c56ec70d6f253656368c9"} Jul 05 12:25:35.750587 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:35.750262 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:35.766230 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:35.766185 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" podStartSLOduration=3.259144026 podStartE2EDuration="3.766173555s" totalImagesPullingTime="507.029529ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:25:35.765320738 +0000 UTC m=+1920.023531217" watchObservedRunningTime="2026-07-05 12:25:35.766173555 +0000 UTC m=+1920.024384049" Jul 05 12:25:40.824659 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.824627 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:25:40.827975 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.827957 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:40.830720 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.830696 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-40cd04-serving-cert\"" Jul 05 12:25:40.836150 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.835861 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:25:40.963731 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.963695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:40.963731 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.963740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:40.963959 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:40.963770 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g5wg\" (UniqueName: \"kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.065162 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.065111 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.065349 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.065182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.065349 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.065202 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5wg\" (UniqueName: \"kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.065349 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:41.065292 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-40cd04-serving-cert: secret "maas-api-e2e-inf-b-40cd04-serving-cert" not found Jul 05 12:25:41.065448 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:25:41.065375 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls podName:72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a nodeName:}" failed. No retries permitted until 2026-07-05 12:25:41.565355389 +0000 UTC m=+1925.823565827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls") pod "maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" (UID: "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a") : secret "maas-api-e2e-inf-b-40cd04-serving-cert" not found Jul 05 12:25:41.065877 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.065861 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.074311 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.074287 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g5wg\" (UniqueName: \"kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.570873 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.570839 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.573353 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.573324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") pod \"maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.739643 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.739606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:41.759672 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.759646 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:25:41.880115 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:41.880090 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:25:41.881976 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:25:41.881942 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f8ef2a_d4e9_49d6_9a41_22ba71f12c0a.slice/crio-5b6e75176ada86a32767e9a6b3663ad749212be42e9421f0a008d625cb924af3 WatchSource:0}: Error finding container 5b6e75176ada86a32767e9a6b3663ad749212be42e9421f0a008d625cb924af3: Status 404 returned error can't find the container with id 5b6e75176ada86a32767e9a6b3663ad749212be42e9421f0a008d625cb924af3 Jul 05 12:25:42.783670 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:42.783631 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" event={"ID":"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a","Type":"ContainerStarted","Data":"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f"} Jul 05 12:25:42.783883 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:42.783692 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" event={"ID":"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a","Type":"ContainerStarted","Data":"5b6e75176ada86a32767e9a6b3663ad749212be42e9421f0a008d625cb924af3"} Jul 05 12:25:42.783883 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:42.783772 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:25:42.799315 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:42.799267 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" podStartSLOduration=2.427357766 podStartE2EDuration="2.79925003s" totalImagesPullingTime="371.892264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:25:42.797969037 +0000 UTC m=+1927.056179497" watchObservedRunningTime="2026-07-05 12:25:42.79925003 +0000 UTC m=+1927.057460502" Jul 05 12:25:48.794036 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:25:48.793995 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:26:50.704484 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:50.704445 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:26:50.705074 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:50.704670 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" podUID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" containerName="maas-api" containerID="cri-o://22015d5699e2d075249719f8db4c5adc4651816f0e9308a86bf0012d6d3a2938" gracePeriod=30 Jul 05 12:26:51.086581 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.086547 2574 generic.go:356] "Generic (PLEG): container finished" podID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" containerID="22015d5699e2d075249719f8db4c5adc4651816f0e9308a86bf0012d6d3a2938" exitCode=0 Jul 05 12:26:51.086745 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.086627 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" event={"ID":"4c26e211-6b4e-46e5-ad9b-e89ce11dd369","Type":"ContainerDied","Data":"22015d5699e2d075249719f8db4c5adc4651816f0e9308a86bf0012d6d3a2938"} Jul 05 12:26:51.147607 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.147577 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:26:51.190936 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.190896 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jw4g\" (UniqueName: \"kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g\") pod \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " Jul 05 12:26:51.191173 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.190939 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") pod \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " Jul 05 12:26:51.191173 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.191012 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca\") pod \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\" (UID: \"4c26e211-6b4e-46e5-ad9b-e89ce11dd369\") " Jul 05 12:26:51.191587 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.191556 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c26e211-6b4e-46e5-ad9b-e89ce11dd369" (UID: "4c26e211-6b4e-46e5-ad9b-e89ce11dd369"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:26:51.193195 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.193171 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c26e211-6b4e-46e5-ad9b-e89ce11dd369" (UID: "4c26e211-6b4e-46e5-ad9b-e89ce11dd369"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:26:51.193279 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.193171 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g" (OuterVolumeSpecName: "kube-api-access-6jw4g") pod "4c26e211-6b4e-46e5-ad9b-e89ce11dd369" (UID: "4c26e211-6b4e-46e5-ad9b-e89ce11dd369"). InnerVolumeSpecName "kube-api-access-6jw4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:26:51.291898 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.291784 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:26:51.291898 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.291835 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jw4g\" (UniqueName: \"kubernetes.io/projected/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-kube-api-access-6jw4g\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:26:51.291898 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:51.291849 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c26e211-6b4e-46e5-ad9b-e89ce11dd369-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:26:52.092317 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.092289 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" Jul 05 12:26:52.092712 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.092284 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v" event={"ID":"4c26e211-6b4e-46e5-ad9b-e89ce11dd369","Type":"ContainerDied","Data":"a369015444e0e295ce9fbb63b73ae3fd3ffe3f1f370c56ec70d6f253656368c9"} Jul 05 12:26:52.092712 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.092418 2574 scope.go:117] "RemoveContainer" containerID="22015d5699e2d075249719f8db4c5adc4651816f0e9308a86bf0012d6d3a2938" Jul 05 12:26:52.113135 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.113111 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:26:52.116638 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.116618 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-1d2467-59f7549659-qqh7v"] Jul 05 12:26:52.244838 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:26:52.244806 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" path="/var/lib/kubelet/pods/4c26e211-6b4e-46e5-ad9b-e89ce11dd369/volumes" Jul 05 12:27:34.328001 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.327965 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:27:34.328607 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.328181 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" podUID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" containerName="maas-api" containerID="cri-o://d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f" gracePeriod=30 Jul 05 12:27:34.576968 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.576943 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:27:34.585540 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.585456 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g5wg\" (UniqueName: \"kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg\") pod \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " Jul 05 12:27:34.585540 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.585530 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca\") pod \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " Jul 05 12:27:34.585737 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.585577 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") pod \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\" (UID: \"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a\") " Jul 05 12:27:34.585929 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.585904 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" (UID: "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:27:34.587717 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.587695 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg" (OuterVolumeSpecName: "kube-api-access-6g5wg") pod "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" (UID: "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a"). InnerVolumeSpecName "kube-api-access-6g5wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:27:34.587800 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.587700 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" (UID: "72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:27:34.686731 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.686687 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:27:34.686731 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.686720 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g5wg\" (UniqueName: \"kubernetes.io/projected/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-kube-api-access-6g5wg\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:27:34.686731 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:34.686729 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:27:35.284632 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.284593 2574 generic.go:356] "Generic (PLEG): container finished" podID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" containerID="d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f" exitCode=0 Jul 05 12:27:35.284860 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.284643 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" Jul 05 12:27:35.284860 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.284675 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" event={"ID":"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a","Type":"ContainerDied","Data":"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f"} Jul 05 12:27:35.284860 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.284728 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb" event={"ID":"72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a","Type":"ContainerDied","Data":"5b6e75176ada86a32767e9a6b3663ad749212be42e9421f0a008d625cb924af3"} Jul 05 12:27:35.284860 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.284754 2574 scope.go:117] "RemoveContainer" containerID="d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f" Jul 05 12:27:35.293515 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.293488 2574 scope.go:117] "RemoveContainer" containerID="d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f" Jul 05 12:27:35.293830 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:27:35.293808 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f\": container with ID starting with d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f not found: ID does not exist" containerID="d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f" Jul 05 12:27:35.293921 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.293837 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f"} err="failed to get container status \"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f\": rpc error: code = NotFound desc = could not find container \"d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f\": container with ID starting with d8acafda542211f7e145f1c5b1ac42af4a2cb3b680424b2b0373e5140f49634f not found: ID does not exist" Jul 05 12:27:35.306081 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.306048 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:27:35.309839 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:35.309812 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-40cd04-6bd649ff75-vttlb"] Jul 05 12:27:36.244575 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:36.244541 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" path="/var/lib/kubelet/pods/72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a/volumes" Jul 05 12:27:44.563707 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.563666 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:27:44.564212 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564196 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" containerName="maas-api" Jul 05 12:27:44.564258 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564216 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" containerName="maas-api" Jul 05 12:27:44.564258 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564240 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" containerName="maas-api" Jul 05 12:27:44.564258 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564248 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" containerName="maas-api" Jul 05 12:27:44.564351 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564326 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c26e211-6b4e-46e5-ad9b-e89ce11dd369" containerName="maas-api" Jul 05 12:27:44.564351 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.564340 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f8ef2a-d4e9-49d6-9a41-22ba71f12c0a" containerName="maas-api" Jul 05 12:27:44.567636 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.567617 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.570368 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.570348 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-c25ded-serving-cert\"" Jul 05 12:27:44.579149 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.579124 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:27:44.672704 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.672660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.672704 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.672709 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.672975 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.672881 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnspp\" (UniqueName: \"kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.773533 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.773489 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnspp\" (UniqueName: \"kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.773724 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.773548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.773724 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.773571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.773724 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:27:44.773662 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-c25ded-serving-cert: secret "maas-api-e2e-api-a-c25ded-serving-cert" not found Jul 05 12:27:44.773896 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:27:44.773732 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls podName:174c2c48-1649-45b1-ae8b-4bbabc0caf8b nodeName:}" failed. No retries permitted until 2026-07-05 12:27:45.273716418 +0000 UTC m=+2049.531926856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls") pod "maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" (UID: "174c2c48-1649-45b1-ae8b-4bbabc0caf8b") : secret "maas-api-e2e-api-a-c25ded-serving-cert" not found Jul 05 12:27:44.774220 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.774204 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:44.786788 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:44.786746 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnspp\" (UniqueName: \"kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:45.278735 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:45.278699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:45.281091 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:45.281062 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") pod \"maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:45.477651 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:45.477616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:45.614080 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:45.614054 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:27:45.616214 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:27:45.616185 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174c2c48_1649_45b1_ae8b_4bbabc0caf8b.slice/crio-2ea624e9c4d61af20c9c52a6af5d0dd021268a9d12c3c5c3b45f9549273015c4 WatchSource:0}: Error finding container 2ea624e9c4d61af20c9c52a6af5d0dd021268a9d12c3c5c3b45f9549273015c4: Status 404 returned error can't find the container with id 2ea624e9c4d61af20c9c52a6af5d0dd021268a9d12c3c5c3b45f9549273015c4 Jul 05 12:27:46.338404 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:46.338307 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" event={"ID":"174c2c48-1649-45b1-ae8b-4bbabc0caf8b","Type":"ContainerStarted","Data":"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d"} Jul 05 12:27:46.338404 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:46.338365 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" event={"ID":"174c2c48-1649-45b1-ae8b-4bbabc0caf8b","Type":"ContainerStarted","Data":"2ea624e9c4d61af20c9c52a6af5d0dd021268a9d12c3c5c3b45f9549273015c4"} Jul 05 12:27:46.338404 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:46.338395 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:46.354708 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:46.354651 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" podStartSLOduration=1.940254817 podStartE2EDuration="2.354630694s" totalImagesPullingTime="414.375877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:27:46.354058197 +0000 UTC m=+2050.612268657" watchObservedRunningTime="2026-07-05 12:27:46.354630694 +0000 UTC m=+2050.612841180" Jul 05 12:27:52.349263 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.349234 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:27:52.594915 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.594883 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:27:52.598563 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.598536 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.601363 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.601297 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-96ff9c-serving-cert\"" Jul 05 12:27:52.608333 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.608307 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:27:52.642730 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.642698 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.642906 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.642744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.642906 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.642828 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5css\" (UniqueName: \"kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.744101 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.744067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.744252 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.744110 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.744252 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.744151 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5css\" (UniqueName: \"kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.744327 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:27:52.744280 2574 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-96ff9c-serving-cert: secret "maas-api-e2e-api-b-96ff9c-serving-cert" not found Jul 05 12:27:52.744366 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:27:52.744352 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls podName:9aae2ba5-7aec-4e3b-92c4-26909baf604f nodeName:}" failed. No retries permitted until 2026-07-05 12:27:53.244332541 +0000 UTC m=+2057.502542997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls") pod "maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" (UID: "9aae2ba5-7aec-4e3b-92c4-26909baf604f") : secret "maas-api-e2e-api-b-96ff9c-serving-cert" not found Jul 05 12:27:52.744879 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.744858 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:52.755282 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:52.755262 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5css\" (UniqueName: \"kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:53.249652 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:53.249605 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:53.252130 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:53.252107 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") pod \"maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:53.509928 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:53.509824 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:53.639148 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:53.639121 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:27:53.641037 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:27:53.641014 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aae2ba5_7aec_4e3b_92c4_26909baf604f.slice/crio-8c37a54aa48cdb37f01a5a6f043bd47d1dbec0af73140c2f6ac03353d68be7be WatchSource:0}: Error finding container 8c37a54aa48cdb37f01a5a6f043bd47d1dbec0af73140c2f6ac03353d68be7be: Status 404 returned error can't find the container with id 8c37a54aa48cdb37f01a5a6f043bd47d1dbec0af73140c2f6ac03353d68be7be Jul 05 12:27:54.380740 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:54.380650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" event={"ID":"9aae2ba5-7aec-4e3b-92c4-26909baf604f","Type":"ContainerStarted","Data":"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1"} Jul 05 12:27:54.380740 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:54.380705 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" event={"ID":"9aae2ba5-7aec-4e3b-92c4-26909baf604f","Type":"ContainerStarted","Data":"8c37a54aa48cdb37f01a5a6f043bd47d1dbec0af73140c2f6ac03353d68be7be"} Jul 05 12:27:54.380740 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:54.380729 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:27:54.397313 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:27:54.397263 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" podStartSLOduration=2.089387658 podStartE2EDuration="2.397248572s" totalImagesPullingTime="307.860914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:27:54.396550055 +0000 UTC m=+2058.654760532" watchObservedRunningTime="2026-07-05 12:27:54.397248572 +0000 UTC m=+2058.655459068" Jul 05 12:28:00.391445 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:00.391368 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:28:05.921513 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:05.921479 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:28:05.922051 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:05.921743 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" podUID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" containerName="maas-api" containerID="cri-o://40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d" gracePeriod=30 Jul 05 12:28:06.166115 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.166092 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:28:06.267012 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.266940 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") pod \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " Jul 05 12:28:06.267012 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.266992 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnspp\" (UniqueName: \"kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp\") pod \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " Jul 05 12:28:06.267209 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.267087 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca\") pod \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\" (UID: \"174c2c48-1649-45b1-ae8b-4bbabc0caf8b\") " Jul 05 12:28:06.267431 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.267408 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "174c2c48-1649-45b1-ae8b-4bbabc0caf8b" (UID: "174c2c48-1649-45b1-ae8b-4bbabc0caf8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:28:06.268988 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.268962 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp" (OuterVolumeSpecName: "kube-api-access-lnspp") pod "174c2c48-1649-45b1-ae8b-4bbabc0caf8b" (UID: "174c2c48-1649-45b1-ae8b-4bbabc0caf8b"). InnerVolumeSpecName "kube-api-access-lnspp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:28:06.269091 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.269032 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "174c2c48-1649-45b1-ae8b-4bbabc0caf8b" (UID: "174c2c48-1649-45b1-ae8b-4bbabc0caf8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:28:06.368822 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.368753 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:06.368822 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.368826 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:06.369071 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.368843 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnspp\" (UniqueName: \"kubernetes.io/projected/174c2c48-1649-45b1-ae8b-4bbabc0caf8b-kube-api-access-lnspp\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:06.434898 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.434861 2574 generic.go:356] "Generic (PLEG): container finished" podID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" containerID="40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d" exitCode=0 Jul 05 12:28:06.435059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.434919 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" Jul 05 12:28:06.435059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.434933 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" event={"ID":"174c2c48-1649-45b1-ae8b-4bbabc0caf8b","Type":"ContainerDied","Data":"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d"} Jul 05 12:28:06.435059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.435001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l" event={"ID":"174c2c48-1649-45b1-ae8b-4bbabc0caf8b","Type":"ContainerDied","Data":"2ea624e9c4d61af20c9c52a6af5d0dd021268a9d12c3c5c3b45f9549273015c4"} Jul 05 12:28:06.435059 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.435025 2574 scope.go:117] "RemoveContainer" containerID="40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d" Jul 05 12:28:06.443498 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.443473 2574 scope.go:117] "RemoveContainer" containerID="40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d" Jul 05 12:28:06.443770 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:28:06.443737 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d\": container with ID starting with 40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d not found: ID does not exist" containerID="40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d" Jul 05 12:28:06.443843 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.443790 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d"} err="failed to get container status \"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d\": rpc error: code = NotFound desc = could not find container \"40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d\": container with ID starting with 40e53fa25cf2c26ee3acb19611800dccc0dec7ac866a6f8d14317a18c671635d not found: ID does not exist" Jul 05 12:28:06.455929 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.455896 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:28:06.459452 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:06.459431 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c25ded-5b7d9b8455-79g2l"] Jul 05 12:28:08.244162 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:08.244127 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" path="/var/lib/kubelet/pods/174c2c48-1649-45b1-ae8b-4bbabc0caf8b/volumes" Jul 05 12:28:12.709779 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:12.709734 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:28:12.710221 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:12.709998 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" podUID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" containerName="maas-api" containerID="cri-o://967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1" gracePeriod=30 Jul 05 12:28:12.955605 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:12.955581 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:28:13.137717 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.137634 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5css\" (UniqueName: \"kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css\") pod \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " Jul 05 12:28:13.137717 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.137681 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") pod \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " Jul 05 12:28:13.137999 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.137728 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca\") pod \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\" (UID: \"9aae2ba5-7aec-4e3b-92c4-26909baf604f\") " Jul 05 12:28:13.138127 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.138106 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9aae2ba5-7aec-4e3b-92c4-26909baf604f" (UID: "9aae2ba5-7aec-4e3b-92c4-26909baf604f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:28:13.139830 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.139803 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9aae2ba5-7aec-4e3b-92c4-26909baf604f" (UID: "9aae2ba5-7aec-4e3b-92c4-26909baf604f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:28:13.139911 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.139824 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css" (OuterVolumeSpecName: "kube-api-access-n5css") pod "9aae2ba5-7aec-4e3b-92c4-26909baf604f" (UID: "9aae2ba5-7aec-4e3b-92c4-26909baf604f"). InnerVolumeSpecName "kube-api-access-n5css". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:28:13.238385 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.238357 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aae2ba5-7aec-4e3b-92c4-26909baf604f-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:13.238385 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.238382 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5css\" (UniqueName: \"kubernetes.io/projected/9aae2ba5-7aec-4e3b-92c4-26909baf604f-kube-api-access-n5css\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:13.238551 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.238393 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aae2ba5-7aec-4e3b-92c4-26909baf604f-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:28:13.473645 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.473610 2574 generic.go:356] "Generic (PLEG): container finished" podID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" containerID="967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1" exitCode=0 Jul 05 12:28:13.473839 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.473652 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" Jul 05 12:28:13.473839 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.473684 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" event={"ID":"9aae2ba5-7aec-4e3b-92c4-26909baf604f","Type":"ContainerDied","Data":"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1"} Jul 05 12:28:13.473839 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.473730 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj" event={"ID":"9aae2ba5-7aec-4e3b-92c4-26909baf604f","Type":"ContainerDied","Data":"8c37a54aa48cdb37f01a5a6f043bd47d1dbec0af73140c2f6ac03353d68be7be"} Jul 05 12:28:13.473839 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.473754 2574 scope.go:117] "RemoveContainer" containerID="967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1" Jul 05 12:28:13.482817 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.482801 2574 scope.go:117] "RemoveContainer" containerID="967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1" Jul 05 12:28:13.483078 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:28:13.483061 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1\": container with ID starting with 967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1 not found: ID does not exist" containerID="967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1" Jul 05 12:28:13.483141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.483089 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1"} err="failed to get container status \"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1\": rpc error: code = NotFound desc = could not find container \"967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1\": container with ID starting with 967ab1b00d2b79dca73c57fedbf279627c1563ac05c7441bd8e542f48b1784a1 not found: ID does not exist" Jul 05 12:28:13.495299 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.495271 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:28:13.499452 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:13.499430 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-96ff9c-cc7bcf76-pzsmj"] Jul 05 12:28:14.243837 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:14.243808 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" path="/var/lib/kubelet/pods/9aae2ba5-7aec-4e3b-92c4-26909baf604f/volumes" Jul 05 12:28:22.779498 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779464 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:28:22.779957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779870 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" containerName="maas-api" Jul 05 12:28:22.779957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779883 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" containerName="maas-api" Jul 05 12:28:22.779957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779899 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" containerName="maas-api" Jul 05 12:28:22.779957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779905 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" containerName="maas-api" Jul 05 12:28:22.779957 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779960 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aae2ba5-7aec-4e3b-92c4-26909baf604f" containerName="maas-api" Jul 05 12:28:22.780131 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.779969 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="174c2c48-1649-45b1-ae8b-4bbabc0caf8b" containerName="maas-api" Jul 05 12:28:22.784247 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.784231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.786843 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.786825 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-6f19c8-serving-cert\"" Jul 05 12:28:22.794636 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.794610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:28:22.821490 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.821460 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.821634 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.821507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.821634 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.821612 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxc6s\" (UniqueName: \"kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.922641 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.922606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxc6s\" (UniqueName: \"kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.922848 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.922675 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.922848 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.922705 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.923402 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.923377 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.925029 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.925008 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:22.929859 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:22.929839 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxc6s\" (UniqueName: \"kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s\") pod \"maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:23.096455 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:23.096374 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:23.222152 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:23.222129 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:28:23.224104 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:28:23.224078 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd8838c_9310_4f46_8371_a17df9010452.slice/crio-ec0ed62a8bbbb548509ce67bdb7754f5fe9ba8069fe0991d12bc1627fbafd079 WatchSource:0}: Error finding container ec0ed62a8bbbb548509ce67bdb7754f5fe9ba8069fe0991d12bc1627fbafd079: Status 404 returned error can't find the container with id ec0ed62a8bbbb548509ce67bdb7754f5fe9ba8069fe0991d12bc1627fbafd079 Jul 05 12:28:23.520561 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:23.520515 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" event={"ID":"edd8838c-9310-4f46-8371-a17df9010452","Type":"ContainerStarted","Data":"ec0ed62a8bbbb548509ce67bdb7754f5fe9ba8069fe0991d12bc1627fbafd079"} Jul 05 12:28:24.526397 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:24.526357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" event={"ID":"edd8838c-9310-4f46-8371-a17df9010452","Type":"ContainerStarted","Data":"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3"} Jul 05 12:28:24.526851 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:24.526439 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:24.543676 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:24.543629 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" podStartSLOduration=2.221866245 podStartE2EDuration="2.543614865s" totalImagesPullingTime="321.74862ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:28:24.542089041 +0000 UTC m=+2088.800299523" watchObservedRunningTime="2026-07-05 12:28:24.543614865 +0000 UTC m=+2088.801825325" Jul 05 12:28:30.537376 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.537348 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:28:30.875023 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.874944 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:28:30.878607 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.878582 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:30.880922 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.880902 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-85c29d-serving-cert\"" Jul 05 12:28:30.888682 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.888656 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:28:30.993136 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.993101 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:30.993303 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.993168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:30.993303 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:30.993207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lxm9\" (UniqueName: \"kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.094281 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.094247 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.094453 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.094295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lxm9\" (UniqueName: \"kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.094453 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.094331 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.094981 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.094957 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.096732 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.096703 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.102354 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.102329 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lxm9\" (UniqueName: \"kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9\") pod \"maas-api-e2e-shared-b-85c29d-774959779-rj2kw\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.190528 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.190503 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:31.317000 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.316977 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:28:31.319159 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:28:31.319129 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ad9d0c_b488_42cd_a16c_0a95aba3f844.slice/crio-9caccac89a832ac3d32abc7a722f68d112bf62b23776cb8f4dbe8d383a120772 WatchSource:0}: Error finding container 9caccac89a832ac3d32abc7a722f68d112bf62b23776cb8f4dbe8d383a120772: Status 404 returned error can't find the container with id 9caccac89a832ac3d32abc7a722f68d112bf62b23776cb8f4dbe8d383a120772 Jul 05 12:28:31.559381 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:31.559294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" event={"ID":"a5ad9d0c-b488-42cd-a16c-0a95aba3f844","Type":"ContainerStarted","Data":"9caccac89a832ac3d32abc7a722f68d112bf62b23776cb8f4dbe8d383a120772"} Jul 05 12:28:32.564904 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:32.564866 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" event={"ID":"a5ad9d0c-b488-42cd-a16c-0a95aba3f844","Type":"ContainerStarted","Data":"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a"} Jul 05 12:28:32.565297 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:32.564971 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:28:32.584024 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:32.583976 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" podStartSLOduration=2.274820877 podStartE2EDuration="2.58396074s" totalImagesPullingTime="309.139863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-05 12:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:28:32.582795689 +0000 UTC m=+2096.841006146" watchObservedRunningTime="2026-07-05 12:28:32.58396074 +0000 UTC m=+2096.842171201" Jul 05 12:28:38.575599 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:28:38.575567 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:33:51.879414 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:51.879380 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:33:51.960419 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:51.960353 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" podUID="edd8838c-9310-4f46-8371-a17df9010452" containerName="maas-api" containerID="cri-o://061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3" gracePeriod=30 Jul 05 12:33:52.208487 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.208463 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:33:52.233861 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.233819 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca\") pod \"edd8838c-9310-4f46-8371-a17df9010452\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " Jul 05 12:33:52.234034 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.233907 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxc6s\" (UniqueName: \"kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s\") pod \"edd8838c-9310-4f46-8371-a17df9010452\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " Jul 05 12:33:52.234034 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.234008 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls\") pod \"edd8838c-9310-4f46-8371-a17df9010452\" (UID: \"edd8838c-9310-4f46-8371-a17df9010452\") " Jul 05 12:33:52.234182 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.234158 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "edd8838c-9310-4f46-8371-a17df9010452" (UID: "edd8838c-9310-4f46-8371-a17df9010452"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:33:52.234328 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.234311 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edd8838c-9310-4f46-8371-a17df9010452-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:33:52.236020 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.235993 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "edd8838c-9310-4f46-8371-a17df9010452" (UID: "edd8838c-9310-4f46-8371-a17df9010452"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:33:52.236110 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.236031 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s" (OuterVolumeSpecName: "kube-api-access-hxc6s") pod "edd8838c-9310-4f46-8371-a17df9010452" (UID: "edd8838c-9310-4f46-8371-a17df9010452"). InnerVolumeSpecName "kube-api-access-hxc6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:33:52.334821 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.334756 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxc6s\" (UniqueName: \"kubernetes.io/projected/edd8838c-9310-4f46-8371-a17df9010452-kube-api-access-hxc6s\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:33:52.334821 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.334813 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edd8838c-9310-4f46-8371-a17df9010452-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:33:52.966389 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.966349 2574 generic.go:356] "Generic (PLEG): container finished" podID="edd8838c-9310-4f46-8371-a17df9010452" containerID="061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3" exitCode=0 Jul 05 12:33:52.966389 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.966391 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" Jul 05 12:33:52.966889 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.966432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" event={"ID":"edd8838c-9310-4f46-8371-a17df9010452","Type":"ContainerDied","Data":"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3"} Jul 05 12:33:52.966889 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.966484 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n" event={"ID":"edd8838c-9310-4f46-8371-a17df9010452","Type":"ContainerDied","Data":"ec0ed62a8bbbb548509ce67bdb7754f5fe9ba8069fe0991d12bc1627fbafd079"} Jul 05 12:33:52.966889 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.966508 2574 scope.go:117] "RemoveContainer" containerID="061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3" Jul 05 12:33:52.975210 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.975192 2574 scope.go:117] "RemoveContainer" containerID="061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3" Jul 05 12:33:52.975491 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:33:52.975462 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3\": container with ID starting with 061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3 not found: ID does not exist" containerID="061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3" Jul 05 12:33:52.975540 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.975499 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3"} err="failed to get container status \"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3\": rpc error: code = NotFound desc = could not find container \"061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3\": container with ID starting with 061494c50102cc032d452e5c4a2572db9bd44325665c364293297633434490a3 not found: ID does not exist" Jul 05 12:33:52.981828 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.981806 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:33:52.989171 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:52.989145 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-6f19c8-d9c69f5b4-7dc2n"] Jul 05 12:33:54.244214 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:33:54.244169 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd8838c-9310-4f46-8371-a17df9010452" path="/var/lib/kubelet/pods/edd8838c-9310-4f46-8371-a17df9010452/volumes" Jul 05 12:34:32.841078 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:32.841034 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:34:32.841630 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:32.841285 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" podUID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" containerName="maas-api" containerID="cri-o://9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a" gracePeriod=30 Jul 05 12:34:33.083419 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.083394 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:34:33.146218 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.146137 2574 generic.go:356] "Generic (PLEG): container finished" podID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" containerID="9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a" exitCode=0 Jul 05 12:34:33.146218 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.146185 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" Jul 05 12:34:33.146401 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.146190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" event={"ID":"a5ad9d0c-b488-42cd-a16c-0a95aba3f844","Type":"ContainerDied","Data":"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a"} Jul 05 12:34:33.146401 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.146294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw" event={"ID":"a5ad9d0c-b488-42cd-a16c-0a95aba3f844","Type":"ContainerDied","Data":"9caccac89a832ac3d32abc7a722f68d112bf62b23776cb8f4dbe8d383a120772"} Jul 05 12:34:33.146401 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.146316 2574 scope.go:117] "RemoveContainer" containerID="9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a" Jul 05 12:34:33.154321 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.154301 2574 scope.go:117] "RemoveContainer" containerID="9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a" Jul 05 12:34:33.154555 ip-10-0-143-202 kubenswrapper[2574]: E0705 12:34:33.154535 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a\": container with ID starting with 9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a not found: ID does not exist" containerID="9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a" Jul 05 12:34:33.154622 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.154569 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a"} err="failed to get container status \"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a\": rpc error: code = NotFound desc = could not find container \"9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a\": container with ID starting with 9414ce124c7b34ffe8cee641e2c11809c4b2c7475c33434aec8080065aefb33a not found: ID does not exist" Jul 05 12:34:33.201065 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.201031 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca\") pod \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " Jul 05 12:34:33.201065 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.201068 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls\") pod \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " Jul 05 12:34:33.201264 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.201096 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lxm9\" (UniqueName: \"kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9\") pod \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\" (UID: \"a5ad9d0c-b488-42cd-a16c-0a95aba3f844\") " Jul 05 12:34:33.201411 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.201388 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5ad9d0c-b488-42cd-a16c-0a95aba3f844" (UID: "a5ad9d0c-b488-42cd-a16c-0a95aba3f844"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 05 12:34:33.203109 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.203088 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9" (OuterVolumeSpecName: "kube-api-access-2lxm9") pod "a5ad9d0c-b488-42cd-a16c-0a95aba3f844" (UID: "a5ad9d0c-b488-42cd-a16c-0a95aba3f844"). InnerVolumeSpecName "kube-api-access-2lxm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 05 12:34:33.203277 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.203255 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5ad9d0c-b488-42cd-a16c-0a95aba3f844" (UID: "a5ad9d0c-b488-42cd-a16c-0a95aba3f844"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 05 12:34:33.302549 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.302505 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-openshift-service-ca\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:34:33.302549 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.302540 2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-maas-api-tls\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:34:33.302549 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.302553 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lxm9\" (UniqueName: \"kubernetes.io/projected/a5ad9d0c-b488-42cd-a16c-0a95aba3f844-kube-api-access-2lxm9\") on node \"ip-10-0-143-202.ec2.internal\" DevicePath \"\"" Jul 05 12:34:33.467707 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.467680 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:34:33.471623 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:33.471600 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-85c29d-774959779-rj2kw"] Jul 05 12:34:34.243402 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:34.243368 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" path="/var/lib/kubelet/pods/a5ad9d0c-b488-42cd-a16c-0a95aba3f844/volumes" Jul 05 12:34:39.706122 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:39.706086 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-674586996d-zfqgn_30d0d8a5-2aed-44b1-a1e6-01aaa4d28939/authorino/0.log" Jul 05 12:34:44.863711 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:44.863679 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b4b8d7d5-jq6qn_24c3a2b1-03a2-4d70-9244-f9b394082c2a/maas-api/0.log" Jul 05 12:34:45.600448 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:45.600418 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-4fmgw_23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30/copy-manifests/0.log" Jul 05 12:34:45.606855 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:45.606832 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-4fmgw_23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30/manager/0.log" Jul 05 12:34:45.981100 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:45.981074 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mcbw5_77ddbe0f-b7d9-469e-bf28-b65db9212a33/postgres/0.log" Jul 05 12:34:47.329222 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:47.329189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-674586996d-zfqgn_30d0d8a5-2aed-44b1-a1e6-01aaa4d28939/authorino/0.log" Jul 05 12:34:47.705468 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:47.705439 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hkf4k_5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244/kuadrant-console-plugin/0.log" Jul 05 12:34:48.808179 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:48.808143 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-gtnpq_837c4af3-92b0-44f9-883d-85668cd234c1/kube-auth-proxy/0.log" Jul 05 12:34:49.852523 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:49.852497 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-488dd_4cd3d94d-0040-4dab-8b3b-bd8e909a4fe4/main/0.log" Jul 05 12:34:50.192013 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:50.191978 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rrwr8_4cdec196-89c4-4567-9059-05c0623ac918/main/0.log" Jul 05 12:34:50.309139 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:50.309110 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-t7hp7_f87c34db-4e93-42de-b846-8fbb83149801/main/0.log" Jul 05 12:34:57.428154 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:57.428122 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-prgqv_73b426e4-95af-454b-a6d5-50e35d34a4fa/global-pull-secret-syncer/0.log" Jul 05 12:34:57.509924 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:57.509896 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jvkdp_e3e98aea-c00e-4411-a3f1-f4d332dd5959/konnectivity-agent/0.log" Jul 05 12:34:57.604237 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:34:57.604204 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-202.ec2.internal_71665d816167a69ba4f61ac61f31e766/haproxy/0.log" Jul 05 12:35:02.669931 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:02.669897 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-674586996d-zfqgn_30d0d8a5-2aed-44b1-a1e6-01aaa4d28939/authorino/0.log" Jul 05 12:35:02.752475 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:02.752447 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hkf4k_5adeb6f1-eee2-4f58-ad0d-3b1ea3fab244/kuadrant-console-plugin/0.log" Jul 05 12:35:05.157745 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.157715 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24zbx_8a1de157-97c4-4256-bf25-9d837858e2af/node-exporter/0.log" Jul 05 12:35:05.180920 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.180894 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24zbx_8a1de157-97c4-4256-bf25-9d837858e2af/kube-rbac-proxy/0.log" Jul 05 12:35:05.205315 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.205289 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-24zbx_8a1de157-97c4-4256-bf25-9d837858e2af/init-textfile/0.log" Jul 05 12:35:05.382084 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.382058 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fk9z2_d495752d-bf11-4d9c-b0c4-3d0dd2937138/kube-rbac-proxy-main/0.log" Jul 05 12:35:05.401728 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.401700 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fk9z2_d495752d-bf11-4d9c-b0c4-3d0dd2937138/kube-rbac-proxy-self/0.log" Jul 05 12:35:05.422151 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.422131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fk9z2_d495752d-bf11-4d9c-b0c4-3d0dd2937138/openshift-state-metrics/0.log" Jul 05 12:35:05.465065 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.465038 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/prometheus/0.log" Jul 05 12:35:05.485254 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.485214 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/config-reloader/0.log" Jul 05 12:35:05.505947 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.505923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/thanos-sidecar/0.log" Jul 05 12:35:05.525639 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.525616 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/kube-rbac-proxy-web/0.log" Jul 05 12:35:05.546982 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.546958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/kube-rbac-proxy/0.log" Jul 05 12:35:05.566031 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.565958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/kube-rbac-proxy-thanos/0.log" Jul 05 12:35:05.587943 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.587919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5db51df0-29ab-4ecb-b550-ebca09ba4eb6/init-config-reloader/0.log" Jul 05 12:35:05.782890 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.782852 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/thanos-query/0.log" Jul 05 12:35:05.805162 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.805131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/kube-rbac-proxy-web/0.log" Jul 05 12:35:05.828863 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.828790 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/kube-rbac-proxy/0.log" Jul 05 12:35:05.848504 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.848477 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/prom-label-proxy/0.log" Jul 05 12:35:05.868844 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.868807 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/kube-rbac-proxy-rules/0.log" Jul 05 12:35:05.890968 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:05.890939 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5b9698864d-5z6b7_a6262d58-6df7-4fac-b93e-54e6a7fe3cbf/kube-rbac-proxy-metrics/0.log" Jul 05 12:35:07.834691 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.834659 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4"] Jul 05 12:35:07.835094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835014 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd8838c-9310-4f46-8371-a17df9010452" containerName="maas-api" Jul 05 12:35:07.835094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835028 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd8838c-9310-4f46-8371-a17df9010452" containerName="maas-api" Jul 05 12:35:07.835094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835040 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" containerName="maas-api" Jul 05 12:35:07.835094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835046 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" containerName="maas-api" Jul 05 12:35:07.835094 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835094 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd8838c-9310-4f46-8371-a17df9010452" containerName="maas-api" Jul 05 12:35:07.835255 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.835104 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ad9d0c-b488-42cd-a16c-0a95aba3f844" containerName="maas-api" Jul 05 12:35:07.838199 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.838183 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:07.840651 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.840629 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dkr2v\"/\"openshift-service-ca.crt\"" Jul 05 12:35:07.841937 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.841912 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dkr2v\"/\"kube-root-ca.crt\"" Jul 05 12:35:07.842081 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.841968 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dkr2v\"/\"default-dockercfg-9cplw\"" Jul 05 12:35:07.847305 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.847281 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4"] Jul 05 12:35:07.904635 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.904597 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-sys\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:07.904862 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.904660 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-podres\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:07.904862 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.904694 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l6tj\" (UniqueName: \"kubernetes.io/projected/c18d9d37-80f1-4190-97dc-67ec4e033495-kube-api-access-6l6tj\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:07.904862 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.904805 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-proc\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:07.905006 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:07.904871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-lib-modules\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.005971 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.005928 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-proc\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006174 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.005980 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-lib-modules\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006174 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006044 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-sys\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006174 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006071 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-proc\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006174 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006086 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-podres\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006174 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-sys\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006365 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l6tj\" (UniqueName: \"kubernetes.io/projected/c18d9d37-80f1-4190-97dc-67ec4e033495-kube-api-access-6l6tj\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006365 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006189 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-lib-modules\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.006365 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.006192 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c18d9d37-80f1-4190-97dc-67ec4e033495-podres\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.013690 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.013658 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l6tj\" (UniqueName: \"kubernetes.io/projected/c18d9d37-80f1-4190-97dc-67ec4e033495-kube-api-access-6l6tj\") pod \"perf-node-gather-daemonset-4slr4\" (UID: \"c18d9d37-80f1-4190-97dc-67ec4e033495\") " pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.148979 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.148887 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:08.233720 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.233692 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-55899fd9dd-m9lhk_a5b27eff-3c6d-4472-8009-2acb2e584f6d/console/0.log" Jul 05 12:35:08.271678 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.271652 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-hkrq2_2afbac7d-2575-439e-b7a5-f66063a6417a/download-server/0.log" Jul 05 12:35:08.290584 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.290487 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4"] Jul 05 12:35:08.293415 ip-10-0-143-202 kubenswrapper[2574]: W0705 12:35:08.293385 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc18d9d37_80f1_4190_97dc_67ec4e033495.slice/crio-7855c5969704bc7d76b1c66d37888e35d5e353a5bc612c4f80e85053d47ba837 WatchSource:0}: Error finding container 7855c5969704bc7d76b1c66d37888e35d5e353a5bc612c4f80e85053d47ba837: Status 404 returned error can't find the container with id 7855c5969704bc7d76b1c66d37888e35d5e353a5bc612c4f80e85053d47ba837 Jul 05 12:35:08.301833 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:08.301804 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" event={"ID":"c18d9d37-80f1-4190-97dc-67ec4e033495","Type":"ContainerStarted","Data":"7855c5969704bc7d76b1c66d37888e35d5e353a5bc612c4f80e85053d47ba837"} Jul 05 12:35:09.310313 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:09.310274 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" event={"ID":"c18d9d37-80f1-4190-97dc-67ec4e033495","Type":"ContainerStarted","Data":"6f95e32009a091258fb1cae3d7ff98114bbc9a8662b66ebdf117e3cb664b1215"} Jul 05 12:35:09.310746 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:09.310337 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:09.325356 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:09.325308 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" podStartSLOduration=2.325293371 podStartE2EDuration="2.325293371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-05 12:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-05 12:35:09.324226206 +0000 UTC m=+2493.582436691" watchObservedRunningTime="2026-07-05 12:35:09.325293371 +0000 UTC m=+2493.583503830" Jul 05 12:35:09.999894 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:09.999863 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n87rx_9a8e5964-b285-4b87-b31d-6a165e12c20b/dns/0.log" Jul 05 12:35:10.026996 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:10.026970 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n87rx_9a8e5964-b285-4b87-b31d-6a165e12c20b/kube-rbac-proxy/0.log" Jul 05 12:35:10.074347 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:10.074319 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f6l9g_c5da800a-b051-4182-92ae-40b4d44daeaf/dns-node-resolver/0.log" Jul 05 12:35:10.633639 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:10.633607 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-646c4846d9-bl7dd_651e7d2b-fea9-419f-bd91-3b3ae586f620/registry/0.log" Jul 05 12:35:10.698879 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:10.698849 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x7tsl_8ab7a494-9ef9-4bb2-b923-43004cfff79f/node-ca/0.log" Jul 05 12:35:11.738166 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:11.738132 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-gtnpq_837c4af3-92b0-44f9-883d-85668cd234c1/kube-auth-proxy/0.log" Jul 05 12:35:12.650534 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:12.650506 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xs6k6_567bbb08-f82e-4ce3-bf69-2c3e236990e2/serve-healthcheck-canary/0.log" Jul 05 12:35:13.121386 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:13.121356 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cl6j9_0e06f685-a431-4af2-adfe-d13f498b1d26/kube-rbac-proxy/0.log" Jul 05 12:35:13.140843 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:13.140816 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cl6j9_0e06f685-a431-4af2-adfe-d13f498b1d26/exporter/0.log" Jul 05 12:35:13.159520 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:13.159494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cl6j9_0e06f685-a431-4af2-adfe-d13f498b1d26/extractor/0.log" Jul 05 12:35:15.324141 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:15.324114 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dkr2v/perf-node-gather-daemonset-4slr4" Jul 05 12:35:15.560784 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:15.560730 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-b4b8d7d5-jq6qn_24c3a2b1-03a2-4d70-9244-f9b394082c2a/maas-api/0.log" Jul 05 12:35:15.799597 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:15.799562 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-4fmgw_23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30/manager/0.log" Jul 05 12:35:15.818617 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:15.818578 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59595c8d98-4fmgw_23771e3a-e3e6-432a-b7ab-9e3c4ff8ac30/copy-manifests/0.log" Jul 05 12:35:15.978133 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:15.978104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mcbw5_77ddbe0f-b7d9-469e-bf28-b65db9212a33/postgres/0.log" Jul 05 12:35:17.194695 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:17.194667 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58544b45f-5k7t7_70457dea-54c4-4d30-b272-4dee1c66eb8b/manager/0.log" Jul 05 12:35:18.028225 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:18.028199 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-gtnpq_837c4af3-92b0-44f9-883d-85668cd234c1/kube-auth-proxy/0.log" Jul 05 12:35:19.575167 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:19.575136 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f99dd8ff-gtnpq_837c4af3-92b0-44f9-883d-85668cd234c1/kube-auth-proxy/0.log" Jul 05 12:35:24.108277 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.108243 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/kube-multus-additional-cni-plugins/0.log" Jul 05 12:35:24.127974 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.127951 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/egress-router-binary-copy/0.log" Jul 05 12:35:24.148955 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.148928 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/cni-plugins/0.log" Jul 05 12:35:24.169009 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.168982 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/bond-cni-plugin/0.log" Jul 05 12:35:24.189292 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.189271 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/routeoverride-cni/0.log" Jul 05 12:35:24.209244 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.209218 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/whereabouts-cni-bincopy/0.log" Jul 05 12:35:24.228338 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.228317 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjljl_50cc9a30-33a9-4375-b546-af0b5832bcfb/whereabouts-cni/0.log" Jul 05 12:35:24.289456 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.289425 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vjs65_ec20056c-246c-4228-be7c-06b6de585a85/kube-multus/0.log" Jul 05 12:35:24.336102 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.336073 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29x85_6f110573-9832-4310-93b6-1bc140a48214/network-metrics-daemon/0.log" Jul 05 12:35:24.355065 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:24.355041 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29x85_6f110573-9832-4310-93b6-1bc140a48214/kube-rbac-proxy/0.log" Jul 05 12:35:25.257050 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.257016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/ovn-controller/0.log" Jul 05 12:35:25.273634 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.273608 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/ovn-acl-logging/0.log" Jul 05 12:35:25.294218 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.294186 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/kube-rbac-proxy-node/0.log" Jul 05 12:35:25.314120 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.314093 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/kube-rbac-proxy-ovn-metrics/0.log" Jul 05 12:35:25.333394 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.333373 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/northd/0.log" Jul 05 12:35:25.351806 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.351721 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/nbdb/0.log" Jul 05 12:35:25.370086 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.370062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/sbdb/0.log" Jul 05 12:35:25.423098 ip-10-0-143-202 kubenswrapper[2574]: I0705 12:35:25.423067 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-djdvv_d0bfec19-86cb-4029-8270-4c50a93a4ef6/ovnkube-controller/0.log"