Jul 16 04:37:16.247491 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 04:37:16.247504 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 04:37:16.247513 ip-10-0-137-187 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 04:37:16.247836 ip-10-0-137-187 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 04:37:26.410303 ip-10-0-137-187 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 04:37:26.410321 ip-10-0-137-187 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 71fd52fe83d647c483d8f8f5d4594a6d -- Jul 16 04:38:56.140436 ip-10-0-137-187 systemd[1]: Starting Kubernetes Kubelet... Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.606211 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612378 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612391 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612398 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612404 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612407 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612410 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.640516 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612413 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612416 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612419 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612422 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612425 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612427 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612430 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612432 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612435 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612438 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612441 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612444 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612447 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612450 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612452 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612461 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612464 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612467 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612470 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.769557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612473 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.688802 ip-10-0-137-187 systemd[1]: Started Kubernetes Kubelet. Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612475 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612478 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612480 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612483 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612485 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612488 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612490 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612492 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612495 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612497 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612500 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612502 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612505 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612507 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612511 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612514 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612517 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612520 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612522 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.927076 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612525 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612527 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612530 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612533 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612535 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612538 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612541 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612543 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612546 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612549 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612552 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612554 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612557 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612560 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612562 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612565 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612567 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612570 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612573 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612575 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.928881 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612577 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612580 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612582 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612585 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612588 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612591 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612593 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612596 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612598 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612601 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612603 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612606 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612608 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612611 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612613 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612616 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612618 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612621 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612625 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612627 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.930216 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612630 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612633 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612637 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612659 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612663 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612666 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612669 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612673 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612677 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612680 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612684 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612686 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612689 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.612693 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613428 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613435 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613438 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613441 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613444 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.930931 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613447 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613449 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613453 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613457 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613460 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613463 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613466 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613469 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613471 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613474 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613476 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613479 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613482 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613485 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613488 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613491 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613494 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613497 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613500 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.931565 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613503 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613506 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613509 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613511 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613514 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613517 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613519 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613522 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613524 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613527 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613529 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613532 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613535 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613537 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613540 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613542 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613545 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613548 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613551 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613553 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.932189 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613556 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613558 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613561 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613563 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613566 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613568 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613571 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613574 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613577 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613579 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613582 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613586 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613588 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613591 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613594 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613597 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613599 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613601 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613604 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613606 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.932826 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613609 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613613 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613616 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613620 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613623 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613626 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613629 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613632 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613634 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613637 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613640 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613642 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613645 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613647 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613650 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613652 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613657 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613660 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613663 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.933435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613665 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613667 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613670 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613672 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613675 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613678 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613681 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613684 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613686 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613689 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613691 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613695 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613698 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613701 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613704 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.613707 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614083 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614088 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614091 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614094 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.934018 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614097 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614101 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614103 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614106 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614110 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614114 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614117 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614119 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614124 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614127 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614130 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614133 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614135 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614138 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614161 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614165 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614168 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614171 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614175 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614178 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.934635 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614181 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614183 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614186 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614189 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614192 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614194 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614197 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614199 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614202 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614205 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614207 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614210 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614212 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614216 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614219 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614222 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614225 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614228 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614230 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.936576 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614233 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614238 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614242 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614245 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614247 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614250 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614252 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614255 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614258 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614260 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614263 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614266 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614268 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614271 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614274 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614276 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614278 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614282 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614284 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614287 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.938957 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614289 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614292 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614294 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614297 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614299 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614301 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614304 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614306 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614310 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614312 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614315 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614318 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614320 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614324 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614328 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614330 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614333 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614335 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614337 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.940301 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614340 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614342 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614345 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614347 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614350 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614353 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614356 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614358 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614360 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614363 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614365 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614368 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614370 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614373 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614375 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614378 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.614380 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614453 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614460 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614465 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614469 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 04:38:56.941404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614473 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614477 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614481 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614485 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614488 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614492 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614497 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614500 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614503 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614507 2573 flags.go:64] FLAG: --cgroup-root="" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614510 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614513 2573 flags.go:64] FLAG: --client-ca-file="" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614515 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614518 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614523 2573 flags.go:64] FLAG: --cluster-domain="" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614525 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614529 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614532 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614535 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614539 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614542 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614545 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614548 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614551 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 16 04:38:56.942576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614554 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614557 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614560 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614562 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614566 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614569 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614572 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614575 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614578 2573 flags.go:64] FLAG: --enable-server="true" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614581 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614584 2573 flags.go:64] FLAG: --event-burst="100" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614587 2573 flags.go:64] FLAG: --event-qps="50" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614590 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614595 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614599 2573 flags.go:64] FLAG: --eviction-hard="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614602 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614605 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614608 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614612 2573 flags.go:64] FLAG: --eviction-soft="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614614 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614617 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614620 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614623 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614626 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614628 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 04:38:56.945226 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614631 2573 flags.go:64] FLAG: --feature-gates="" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614635 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614638 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614642 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614645 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614647 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614650 2573 flags.go:64] FLAG: --help="false" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614653 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-137-187.ec2.internal" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614656 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614659 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614662 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614665 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614668 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614671 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614673 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614676 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614679 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614682 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614685 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614687 2573 flags.go:64] FLAG: --kube-reserved="" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614691 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614695 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614698 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614701 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 04:38:56.947081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614704 2573 flags.go:64] FLAG: --lock-file="" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614707 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614710 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614713 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614717 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614720 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614723 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614726 2573 flags.go:64] FLAG: --logging-format="text" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614729 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614732 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614735 2573 flags.go:64] FLAG: --manifest-url="" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614738 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614742 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614745 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614749 2573 flags.go:64] FLAG: --max-pods="110" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614752 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614755 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614758 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614761 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614764 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614767 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614770 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614776 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614779 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614782 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 04:38:56.949896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614785 2573 flags.go:64] FLAG: --pod-cidr="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614788 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614794 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614798 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614801 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614805 2573 flags.go:64] FLAG: --port="10250" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614808 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614811 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01e6f4fd8cd13fc8e" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614814 2573 flags.go:64] FLAG: --qos-reserved="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614817 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614819 2573 flags.go:64] FLAG: --register-node="true" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614822 2573 flags.go:64] FLAG: --register-with-taints="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614826 2573 flags.go:64] FLAG: --registry-burst="10" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614828 2573 flags.go:64] FLAG: --registry-qps="5" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614831 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614834 2573 flags.go:64] FLAG: --reserved-memory="" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614838 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614841 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614844 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614847 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614850 2573 flags.go:64] FLAG: --runonce="false" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614853 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614856 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614859 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614862 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 04:38:56.952001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614864 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614867 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614870 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614873 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614876 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614879 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614881 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614884 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614887 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614890 2573 flags.go:64] FLAG: --system-cgroups="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614895 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614900 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614904 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614906 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614910 2573 flags.go:64] FLAG: --tls-min-version="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614913 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614916 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614918 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614921 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614924 2573 flags.go:64] FLAG: --v="2" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614928 2573 flags.go:64] FLAG: --version="false" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614932 2573 flags.go:64] FLAG: --vmodule="" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614936 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.614939 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 04:38:56.962194 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615068 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615072 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615075 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615080 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615083 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615086 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615089 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615092 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615094 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615097 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615100 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615102 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615104 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615107 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615109 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615112 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615115 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615117 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615121 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.963443 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615123 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615127 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615129 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615132 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615134 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615138 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615154 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615157 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615160 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615163 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615165 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615168 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615170 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615173 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615175 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615178 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615181 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615184 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615186 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615189 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.973631 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615192 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615194 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615197 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615199 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615202 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615205 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615209 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615212 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615215 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615217 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615220 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615223 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615226 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615230 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615233 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615236 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615238 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615241 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615243 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615246 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.974518 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615248 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615251 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615253 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615256 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615258 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615261 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615263 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615266 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615269 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615271 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615274 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615276 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615279 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615281 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615284 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615287 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615289 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615291 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615294 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615296 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.975621 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615299 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615301 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615304 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615308 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615310 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615314 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615317 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615320 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615322 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615325 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615327 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615330 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615332 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615335 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615338 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615341 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615345 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615347 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615350 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.976521 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.615353 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.616024 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.624483 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.624499 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.624513 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.624519 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624579 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624585 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624589 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624593 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624596 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624599 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624602 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624604 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624607 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.977403 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624609 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624612 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624615 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624618 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624621 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624623 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624626 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624628 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624633 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624636 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624638 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624641 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624644 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624647 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624649 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624652 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624656 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624658 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624662 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624664 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.978504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624667 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624669 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624672 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624674 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624677 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624680 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624682 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624685 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624688 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624690 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624692 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624695 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624698 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624700 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624703 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624705 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624708 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624711 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624714 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624716 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.979384 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624720 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624723 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624725 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624728 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624731 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624733 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624735 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624738 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624742 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624744 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624747 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624750 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624752 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624755 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624757 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624760 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624763 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624765 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624768 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624770 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.980279 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624773 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624776 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624778 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624781 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624784 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624786 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624789 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624791 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624794 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624798 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624802 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624804 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624808 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624811 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624813 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624815 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624818 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624820 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624823 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624825 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.981171 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624829 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624832 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624835 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624838 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624842 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624845 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624849 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624853 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624856 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.624858 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.624863 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625000 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625006 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625009 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 04:38:56.982052 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625012 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625015 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625018 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625021 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625025 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625028 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625031 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625034 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625036 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625039 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625043 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625046 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625049 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625053 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625056 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625059 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625062 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625065 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625067 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 04:38:56.982711 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625072 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625075 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625077 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625080 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625082 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625085 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625087 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625090 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625093 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625095 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625098 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625100 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625103 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625105 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625108 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625110 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625113 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625115 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625118 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 04:38:56.983598 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625120 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625122 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625125 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625128 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625133 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625135 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625138 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625140 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625157 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625160 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625163 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625165 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625168 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625172 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625175 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625177 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625180 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625182 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625185 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625188 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 04:38:56.984504 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625190 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625193 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625195 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625198 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625201 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625203 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625206 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625208 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625210 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625213 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625216 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625218 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625221 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625223 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625226 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625229 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625235 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625238 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625241 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625245 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 04:38:56.985764 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625248 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625251 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625254 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625257 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625259 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625263 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625266 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625269 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625271 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625274 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625276 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625279 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625281 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625284 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625286 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625289 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625291 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 04:38:56.986672 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:56.625293 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.625298 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.625401 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.627741 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.629354 2573 server.go:1019] "Starting client certificate rotation" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.629462 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.629531 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.634187 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.636277 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.647284 2573 log.go:25] "Validated CRI v1 runtime API" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.651428 2573 log.go:25] "Validated CRI v1 image API" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.654280 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 04:38:56.987551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.657740 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c148e73d-5508-4703-ae2e-2457d4b283c6:/dev/nvme0n1p4 daec841b-44c2-4200-8111-8933792be16b:/dev/nvme0n1p3] Jul 16 04:38:56.988448 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.657759 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 04:38:56.988448 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.663646 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.664422 2573 manager.go:216] Machine: {Timestamp:2026-07-16 04:38:56.662853397 +0000 UTC m=+0.389223689 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099510 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fe0f862c8a0b1c19f19328cb15e38 SystemUUID:ec2fe0f8-62c8-a0b1-c19f-19328cb15e38 BootID:71fd52fe-83d6-47c4-83d8-f8f5d4594a6d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:43:23:1f:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:43:23:1f:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:41:63:e7:2b:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.664529 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.664637 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.668979 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.669010 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-187.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.670109 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.670126 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.670136 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.671086 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.672517 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.675864 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676761 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676854 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676904 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676929 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676952 2573 kubelet.go:398] "Adding apiserver pod source" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.676966 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 04:38:57.040339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.677614 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.677642 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.680389 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.682947 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.682984 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683216 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683230 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683236 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683243 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683251 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683257 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683851 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683857 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683863 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683869 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683877 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.683893 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.684824 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.684831 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.687988 2573 server.go:1268] "Started kubelet" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.688067 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.688127 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.688737 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.689448 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.690848 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.692710 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.699014 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.699406 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.699554 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700245 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700247 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700269 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700341 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700371 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700872 2573 factory.go:55] Registering systemd factory Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.700890 2573 factory.go:223] Registration of the systemd container factory successfully Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701171 2573 factory.go:153] Registering CRI-O factory Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701185 2573 factory.go:223] Registration of the crio container factory successfully Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701251 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701274 2573 factory.go:103] Registering Raw factory Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701291 2573 manager.go:1196] Started watching for new ooms in manager Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.701711 2573 manager.go:349] Starting recovery of all containers Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.703228 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.703242 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.703230 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 04:38:57.041373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.703315 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-187.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.703352 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-187.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.703233 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c2ab49fecab2e4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-16 04:38:56.687796964 +0000 UTC m=+0.414167254,LastTimestamp:2026-07-16 04:38:56.687796964 +0000 UTC m=+0.414167254,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.716455 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7r9h9" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.721945 2573 manager.go:354] Recovery completed Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.723596 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7r9h9" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.726402 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.729697 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.729724 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.729735 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.730377 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.730393 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.730413 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.730405 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.731385 2573 policy_none.go:49] "None policy: Start" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.731401 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.731414 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.732249 2573 policy_none.go:47] "Start" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.732402 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c2ab4a014a3f52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-187.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-16 04:38:56.729710418 +0000 UTC m=+0.456080710,LastTimestamp:2026-07-16 04:38:56.729710418 +0000 UTC m=+0.456080710,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.733272 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.733295 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.733309 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.787506 2573 manager.go:337] "Starting Device Plugin manager" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.787564 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.787575 2573 server.go:87] "Starting device plugin registration server" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.787818 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.787827 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.787988 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.788079 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.788089 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.788539 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.788590 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.826470 2573 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2607 into the system container \"/system.slice\": " Jul 16 04:38:57.045386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.888658 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.889711 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.889747 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.889757 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.889784 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:56.900402 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.900423 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-187.ec2.internal\": node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:56.931570 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.046955 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.031987 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.172648 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.132398 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.241025 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.232669 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.367765 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.333196 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.434645 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.433664 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.505631 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.506888 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.506935 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.506964 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.507005 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.509960 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:38:57.559318 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.533747 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.607516 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal"] Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.607633 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.609601 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.609634 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.609645 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.612167 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.612347 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.612387 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.615402 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.615431 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.615442 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.616451 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.616480 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.616492 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.619067 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.619095 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.625194 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientMemory" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.625227 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.625249 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeHasSufficientPID" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.628663 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.628858 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 04:38:57.631248 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.628854 2573 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a162f57d183f648a0847f28e16979aed-4f7be0ade2dab6eb.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-137-187.ec2.internal.18c2ab4a014a887d\": read tcp 10.0.137.187:35010->3.232.152.147:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-137-187.ec2.internal.18c2ab4a014a887d default 5856 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-187.ec2.internal,UID:ip-10-0-137-187.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-187.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-187.ec2.internal,},FirstTimestamp:2026-07-16 04:38:56 +0000 UTC,LastTimestamp:2026-07-16 04:38:57.609638829 +0000 UTC m=+1.336009121,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-187.ec2.internal,}" Jul 16 04:38:57.634725 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.633809 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.708371 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.704463 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.708371 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.704502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4a62ec9b3b57c8a82085d965555b5bf-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"c4a62ec9b3b57c8a82085d965555b5bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.708371 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.704529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.782046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.727271 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 04:33:57 +0000 UTC" deadline="2028-01-22 03:15:00.515312118 +0000 UTC" Jul 16 04:38:57.782046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.727304 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13318h36m2.788011519s" Jul 16 04:38:57.782046 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.734546 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805431 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805456 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805516 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4a62ec9b3b57c8a82085d965555b5bf-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"c4a62ec9b3b57c8a82085d965555b5bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805578 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4a62ec9b3b57c8a82085d965555b5bf-config\") pod \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" (UID: \"c4a62ec9b3b57c8a82085d965555b5bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.805591 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/efb47399fcbe75804be2359f043ba9c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" (UID: \"efb47399fcbe75804be2359f043ba9c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:57.855890 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.835528 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.986435 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:57.936269 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-187.ec2.internal\" not found" Jul 16 04:38:57.986435 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:57.967333 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:38:58.000638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.000610 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.009160 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.009120 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.013517 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.013494 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:38:58.015851 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.015831 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.016719 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.016705 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.026455 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.026436 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:38:58.028563 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.028546 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.030280 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.030264 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:38:58.030325 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.030310 2573 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-187.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.032235 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.032221 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 04:38:58.032279 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.032260 2573 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.034548 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.033988 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" Jul 16 04:38:58.064688 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.064655 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:38:58.296195 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.296075 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:38:58.678573 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.678533 2573 apiserver.go:52] "Watching apiserver" Jul 16 04:38:58.684811 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.684783 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 04:38:58.685927 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.685888 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vpthh","kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84","openshift-dns/node-resolver-7rqwt","openshift-multus/multus-additional-cni-plugins-j8gh6","openshift-multus/multus-zpjzr","openshift-multus/network-metrics-daemon-ccprx","openshift-ovn-kubernetes/ovnkube-node-7c4d9","kube-system/konnectivity-agent-tq4p6","openshift-cluster-node-tuning-operator/tuned-qnfrm","openshift-image-registry/node-ca-kgbzl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal","openshift-network-diagnostics/network-check-target-4pb7f","openshift-network-operator/iptables-alerter-jwqzp"] Jul 16 04:38:58.688587 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.688565 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.688703 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.688667 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:38:58.690807 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.690789 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:38:58.690880 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.690858 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:38:58.696046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.696012 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.698490 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.698461 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.698954 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.698931 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 04:38:58.699058 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.699009 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s2s5z\"" Jul 16 04:38:58.699113 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.699047 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.699347 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.699331 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.700959 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.700938 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.701059 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.700970 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.701059 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.700974 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v8sr4\"" Jul 16 04:38:58.701059 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.701007 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 04:38:58.701624 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.701602 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.703934 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.703919 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.704372 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704352 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.704464 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704412 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 04:38:58.704464 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704427 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 04:38:58.704554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704463 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-62bdd\"" Jul 16 04:38:58.704554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704549 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.704627 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.704561 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 04:38:58.705963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.705940 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 04:38:58.706304 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.706292 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.707652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.707633 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lcn5n\"" Jul 16 04:38:58.708584 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.708562 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.709026 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.708651 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:38:58.709195 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.709180 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 04:38:58.709259 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.709243 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 04:38:58.709356 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.709334 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t7bbd\"" Jul 16 04:38:58.711106 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.711085 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.712155 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.712271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:38:58.712271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-sys-fs\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712235 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a26870-a25b-474e-a245-ba01fcd77b05-tmp-dir\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.712271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cnibin\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712328 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712498 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3a26870-a25b-474e-a245-ba01fcd77b05-hosts-file\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.712498 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712462 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpvs8\" (UniqueName: \"kubernetes.io/projected/e6d40b44-4d07-47f8-9804-d474de7ee666-kube-api-access-hpvs8\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq8kk\" (UniqueName: \"kubernetes.io/projected/c3a26870-a25b-474e-a245-ba01fcd77b05-kube-api-access-hq8kk\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712595 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjktf\" (UniqueName: \"kubernetes.io/projected/ab0b8cd2-0e23-41ac-b183-41b735183d6e-kube-api-access-hjktf\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712612 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrg8j\" (UniqueName: \"kubernetes.io/projected/03421519-f85c-4e08-aa29-5b1f437f72bf-kube-api-access-zrg8j\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-socket-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-registration-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-device-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-etc-selinux\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.712959 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.712744 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-os-release\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.713474 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.713450 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.713583 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.713497 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w926s\"" Jul 16 04:38:58.713583 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.713510 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 04:38:58.713739 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.713725 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.713994 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.713978 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.716394 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.716376 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.717769 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:58.717731 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb47399fcbe75804be2359f043ba9c0.slice/crio-dbf5b0d7b0606a59cda2a6adf485e3307ba00dee9a8b5dfb7e17a713aee1863b WatchSource:0}: Error finding container dbf5b0d7b0606a59cda2a6adf485e3307ba00dee9a8b5dfb7e17a713aee1863b: Status 404 returned error can't find the container with id dbf5b0d7b0606a59cda2a6adf485e3307ba00dee9a8b5dfb7e17a713aee1863b Jul 16 04:38:58.718884 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.718865 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.719015 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.718999 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.719795 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.719774 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 04:38:58.720091 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720072 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.720260 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720244 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 04:38:58.720357 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720341 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.720421 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-45lbx\"" Jul 16 04:38:58.720543 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720521 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.720649 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.720633 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dltjz\"" Jul 16 04:38:58.724631 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.723979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 04:38:58.726312 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:58.726279 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a62ec9b3b57c8a82085d965555b5bf.slice/crio-c31d3f2ac95bbc808600fea8348b4d2b1b50b905d9801f1afef8ed176bd01494 WatchSource:0}: Error finding container c31d3f2ac95bbc808600fea8348b4d2b1b50b905d9801f1afef8ed176bd01494: Status 404 returned error can't find the container with id c31d3f2ac95bbc808600fea8348b4d2b1b50b905d9801f1afef8ed176bd01494 Jul 16 04:38:58.726758 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.726725 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 04:38:58.726835 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.726763 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hd96j\"" Jul 16 04:38:58.726896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.726866 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 04:38:58.726949 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.726899 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 04:38:58.728036 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.727934 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 04:38:58.728614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.728583 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 04:38:58.729291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.729263 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:38:58.801671 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.801622 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 04:38:58.813559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftfwc\" (UniqueName: \"kubernetes.io/projected/db73ec87-38ea-4629-a015-bdb64c988d73-kube-api-access-ftfwc\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.813559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-dbus\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-env-overrides\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-os-release\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-netns\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-etc-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cnibin\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813757 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-multus-daemon-config\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.813775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813748 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-os-release\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-lib-modules\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813809 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cnibin\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-config\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3a26870-a25b-474e-a245-ba01fcd77b05-hosts-file\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813909 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-iptables-alerter-script\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813926 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-kubernetes\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltm7l\" (UniqueName: \"kubernetes.io/projected/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-kube-api-access-ltm7l\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813983 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3a26870-a25b-474e-a245-ba01fcd77b05-hosts-file\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.813991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.813991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814028 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpvs8\" (UniqueName: \"kubernetes.io/projected/e6d40b44-4d07-47f8-9804-d474de7ee666-kube-api-access-hpvs8\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814051 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq8kk\" (UniqueName: \"kubernetes.io/projected/c3a26870-a25b-474e-a245-ba01fcd77b05-kube-api-access-hq8kk\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-cnibin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814192 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-cni-binary-copy\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814208 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-etc-kubernetes\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-modprobe-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.814378 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814364 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-socket-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-registration-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814412 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-etc-selinux\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814490 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-registration-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a26870-a25b-474e-a245-ba01fcd77b05-tmp-dir\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814563 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-etc-selinux\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-multus\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814562 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-socket-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814610 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-systemd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-node-log\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814709 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.814773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-hostroot\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814780 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-system-cni-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814853 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-var-lib-kubelet\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.814879 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814884 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-tmp\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814922 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fq2\" (UniqueName: \"kubernetes.io/projected/945aa51d-2a67-49f2-b40e-c4829c655f85-kube-api-access-p2fq2\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjktf\" (UniqueName: \"kubernetes.io/projected/ab0b8cd2-0e23-41ac-b183-41b735183d6e-kube-api-access-hjktf\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.814996 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3a26870-a25b-474e-a245-ba01fcd77b05-tmp-dir\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815005 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-conf-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815122 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-multus-certs\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-device-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.815234 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:38:59.315160881 +0000 UTC m=+3.041531175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815248 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-device-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-kubelet-config\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-host-slash\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.815511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59q72\" (UniqueName: \"kubernetes.io/projected/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-kube-api-access-59q72\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815417 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-systemd\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815440 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-log-socket\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-netd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovn-node-metrics-cert\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815621 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-sys-fs\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysconfig\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-tuned\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-bin\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815714 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-k8s-cni-cncf-io\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815737 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-sys-fs\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-bin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-kubelet\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815905 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab0b8cd2-0e23-41ac-b183-41b735183d6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.816062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-run\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815941 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-sys\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.815983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-serviceca\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-slash\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxwcd\" (UniqueName: \"kubernetes.io/projected/d32e28d9-9cde-4522-af4b-bac6b04ab850-kube-api-access-xxwcd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816055 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-os-release\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-socket-dir-parent\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816099 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-netns\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816118 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-kubelet\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-var-lib-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-ovn\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrg8j\" (UniqueName: \"kubernetes.io/projected/03421519-f85c-4e08-aa29-5b1f437f72bf-kube-api-access-zrg8j\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-system-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816357 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6d40b44-4d07-47f8-9804-d474de7ee666-kubelet-dir\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-agent-certs\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.816652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-konnectivity-ca\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-host\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816441 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-script-lib\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816465 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-host\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-systemd-units\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-conf\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.817174 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.816910 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab0b8cd2-0e23-41ac-b183-41b735183d6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.821525 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.821496 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 04:38:58.821575 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.821545 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:38:58.821575 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.821563 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:38:58.821575 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.821572 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:38:58.821683 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.821617 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:38:59.321604854 +0000 UTC m=+3.047975133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:38:58.825249 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.825205 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq8kk\" (UniqueName: \"kubernetes.io/projected/c3a26870-a25b-474e-a245-ba01fcd77b05-kube-api-access-hq8kk\") pod \"node-resolver-7rqwt\" (UID: \"c3a26870-a25b-474e-a245-ba01fcd77b05\") " pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:58.825370 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.825265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrg8j\" (UniqueName: \"kubernetes.io/projected/03421519-f85c-4e08-aa29-5b1f437f72bf-kube-api-access-zrg8j\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:58.825412 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.825383 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjktf\" (UniqueName: \"kubernetes.io/projected/ab0b8cd2-0e23-41ac-b183-41b735183d6e-kube-api-access-hjktf\") pod \"multus-additional-cni-plugins-j8gh6\" (UID: \"ab0b8cd2-0e23-41ac-b183-41b735183d6e\") " pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:58.825454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.825384 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpvs8\" (UniqueName: \"kubernetes.io/projected/e6d40b44-4d07-47f8-9804-d474de7ee666-kube-api-access-hpvs8\") pod \"aws-ebs-csi-driver-node-67b84\" (UID: \"e6d40b44-4d07-47f8-9804-d474de7ee666\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:58.857006 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.856913 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rgsgt" Jul 16 04:38:58.864557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.864532 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rgsgt" Jul 16 04:38:58.901264 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.901227 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 04:38:58.917428 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-env-overrides\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917428 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-netns\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917446 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-etc-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-multus-daemon-config\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917481 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-lib-modules\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917540 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-netns\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917548 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-etc-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-config\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917636 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-iptables-alerter-script\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917667 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-kubernetes\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltm7l\" (UniqueName: \"kubernetes.io/projected/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-kube-api-access-ltm7l\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917709 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-lib-modules\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917726 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-cnibin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917766 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917787 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-cni-binary-copy\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-etc-kubernetes\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917851 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-modprobe-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917882 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.917963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-multus\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.917822 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-cnibin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918066 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918085 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-etc-kubernetes\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918113 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-systemd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918232 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-modprobe-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-systemd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918255 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-config\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-multus-daemon-config\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918301 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-multus\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-node-log\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918316 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-node-log\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-hostroot\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918353 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db73ec87-38ea-4629-a015-bdb64c988d73-cni-binary-copy\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-kubernetes\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918375 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.918489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-var-lib-kubelet\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918402 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-hostroot\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-tmp\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918444 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-d\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fq2\" (UniqueName: \"kubernetes.io/projected/945aa51d-2a67-49f2-b40e-c4829c655f85-kube-api-access-p2fq2\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918497 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-conf-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918506 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-var-lib-kubelet\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-multus-certs\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918540 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-kubelet-config\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918542 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-conf-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918558 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-host-slash\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918572 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-multus-certs\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918575 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59q72\" (UniqueName: \"kubernetes.io/projected/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-kube-api-access-59q72\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-systemd\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918624 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-host-slash\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-log-socket\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918673 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-systemd\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918672 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-netd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.919354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918677 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-kubelet-config\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918702 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-log-socket\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918739 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovn-node-metrics-cert\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918766 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-netd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918792 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysconfig\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-tuned\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918865 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-env-overrides\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-bin\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918893 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysconfig\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-k8s-cni-cncf-io\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918914 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-cni-bin\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-bin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-kubelet\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-run\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918988 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-cni-bin\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.918957 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-k8s-cni-cncf-io\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-sys\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919010 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-var-lib-kubelet\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.920323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-serviceca\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919051 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-run\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919064 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-sys\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-slash\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxwcd\" (UniqueName: \"kubernetes.io/projected/d32e28d9-9cde-4522-af4b-bac6b04ab850-kube-api-access-xxwcd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919127 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-os-release\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919133 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-slash\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-socket-dir-parent\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919181 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-netns\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919227 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-host-run-netns\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919227 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-os-release\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919273 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-multus-socket-dir-parent\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-kubelet\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919357 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-var-lib-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-ovn\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-kubelet\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-run-ovn\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919455 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-host-run-ovn-kubernetes\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919427 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-var-lib-openvswitch\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919505 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-serviceca\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-system-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919553 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-agent-certs\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-konnectivity-ca\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919606 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-host\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919626 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db73ec87-38ea-4629-a015-bdb64c988d73-system-cni-dir\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919697 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-host\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919633 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-script-lib\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-host\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919760 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-iptables-alerter-script\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-systemd-units\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919816 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d32e28d9-9cde-4522-af4b-bac6b04ab850-systemd-units\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-conf\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919861 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-host\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftfwc\" (UniqueName: \"kubernetes.io/projected/db73ec87-38ea-4629-a015-bdb64c988d73-kube-api-access-ftfwc\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:58.921935 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.919905 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-dbus\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.919946 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.920000 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-dbus\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:58.920006 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:38:59.419988302 +0000 UTC m=+3.146358596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.920113 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-sysctl-conf\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.920184 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovnkube-script-lib\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.920633 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-konnectivity-ca\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.921207 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-tmp\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.921380 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/945aa51d-2a67-49f2-b40e-c4829c655f85-etc-tuned\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.921773 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d32e28d9-9cde-4522-af4b-bac6b04ab850-ovn-node-metrics-cert\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.922484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.922200 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd48f9f6-380b-4ab0-b5b1-02341b0d9801-agent-certs\") pod \"konnectivity-agent-tq4p6\" (UID: \"dd48f9f6-380b-4ab0-b5b1-02341b0d9801\") " pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:58.927109 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.927077 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltm7l\" (UniqueName: \"kubernetes.io/projected/053d5cd9-8f78-4549-bdcc-5139d8b44bd1-kube-api-access-ltm7l\") pod \"node-ca-kgbzl\" (UID: \"053d5cd9-8f78-4549-bdcc-5139d8b44bd1\") " pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:58.927263 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.927243 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fq2\" (UniqueName: \"kubernetes.io/projected/945aa51d-2a67-49f2-b40e-c4829c655f85-kube-api-access-p2fq2\") pod \"tuned-qnfrm\" (UID: \"945aa51d-2a67-49f2-b40e-c4829c655f85\") " pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:58.927308 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.927270 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59q72\" (UniqueName: \"kubernetes.io/projected/8c038c5b-915b-4ef6-a784-d55fdd3ad0f5-kube-api-access-59q72\") pod \"iptables-alerter-jwqzp\" (UID: \"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5\") " pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:58.927628 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.927608 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxwcd\" (UniqueName: \"kubernetes.io/projected/d32e28d9-9cde-4522-af4b-bac6b04ab850-kube-api-access-xxwcd\") pod \"ovnkube-node-7c4d9\" (UID: \"d32e28d9-9cde-4522-af4b-bac6b04ab850\") " pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:58.930453 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:58.930432 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftfwc\" (UniqueName: \"kubernetes.io/projected/db73ec87-38ea-4629-a015-bdb64c988d73-kube-api-access-ftfwc\") pod \"multus-zpjzr\" (UID: \"db73ec87-38ea-4629-a015-bdb64c988d73\") " pod="openshift-multus/multus-zpjzr" Jul 16 04:38:59.019894 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.019847 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" Jul 16 04:38:59.027509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.027483 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7rqwt" Jul 16 04:38:59.030815 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.030788 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d40b44_4d07_47f8_9804_d474de7ee666.slice/crio-db559e1da4378cfbe11534a239f934d1dd2e8cc2d65d0b63e518991c6108e815 WatchSource:0}: Error finding container db559e1da4378cfbe11534a239f934d1dd2e8cc2d65d0b63e518991c6108e815: Status 404 returned error can't find the container with id db559e1da4378cfbe11534a239f934d1dd2e8cc2d65d0b63e518991c6108e815 Jul 16 04:38:59.035722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.035694 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" Jul 16 04:38:59.036185 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.036159 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a26870_a25b_474e_a245_ba01fcd77b05.slice/crio-d186cdcb79eff9f8d7a3c9ccbad7ddcec06aff1026039d27fa647805ba9ed27e WatchSource:0}: Error finding container d186cdcb79eff9f8d7a3c9ccbad7ddcec06aff1026039d27fa647805ba9ed27e: Status 404 returned error can't find the container with id d186cdcb79eff9f8d7a3c9ccbad7ddcec06aff1026039d27fa647805ba9ed27e Jul 16 04:38:59.040120 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.040104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zpjzr" Jul 16 04:38:59.045270 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.045215 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0b8cd2_0e23_41ac_b183_41b735183d6e.slice/crio-7e6c0f8486cb3b9d1ed907baef196fa2ce0ddbf7ce8d1b70c09ef4c474582c5d WatchSource:0}: Error finding container 7e6c0f8486cb3b9d1ed907baef196fa2ce0ddbf7ce8d1b70c09ef4c474582c5d: Status 404 returned error can't find the container with id 7e6c0f8486cb3b9d1ed907baef196fa2ce0ddbf7ce8d1b70c09ef4c474582c5d Jul 16 04:38:59.047511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.047490 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:38:59.051393 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.051357 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb73ec87_38ea_4629_a015_bdb64c988d73.slice/crio-bc07a3f7ab1d7ce581b16e65c007dc9d8988ca85eb276d589a1fe91b2a7a7416 WatchSource:0}: Error finding container bc07a3f7ab1d7ce581b16e65c007dc9d8988ca85eb276d589a1fe91b2a7a7416: Status 404 returned error can't find the container with id bc07a3f7ab1d7ce581b16e65c007dc9d8988ca85eb276d589a1fe91b2a7a7416 Jul 16 04:38:59.053406 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.053393 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jwqzp" Jul 16 04:38:59.056673 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.056652 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd48f9f6_380b_4ab0_b5b1_02341b0d9801.slice/crio-4ac64ba89968c0fd82a1023152fc2c2e5fc07df4c06d7d862cb4149bd7487f46 WatchSource:0}: Error finding container 4ac64ba89968c0fd82a1023152fc2c2e5fc07df4c06d7d862cb4149bd7487f46: Status 404 returned error can't find the container with id 4ac64ba89968c0fd82a1023152fc2c2e5fc07df4c06d7d862cb4149bd7487f46 Jul 16 04:38:59.059530 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.059444 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" Jul 16 04:38:59.062366 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.062342 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c038c5b_915b_4ef6_a784_d55fdd3ad0f5.slice/crio-21af05ca60e25654afd523722f69c97a12de83a6b66e10bbc34fd5331db0d668 WatchSource:0}: Error finding container 21af05ca60e25654afd523722f69c97a12de83a6b66e10bbc34fd5331db0d668: Status 404 returned error can't find the container with id 21af05ca60e25654afd523722f69c97a12de83a6b66e10bbc34fd5331db0d668 Jul 16 04:38:59.065706 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.065686 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgbzl" Jul 16 04:38:59.068102 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.068081 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945aa51d_2a67_49f2_b40e_c4829c655f85.slice/crio-7ae26d960ded9cc953c95f6395626d4c4173bf0dff65d36ac837fada387efead WatchSource:0}: Error finding container 7ae26d960ded9cc953c95f6395626d4c4173bf0dff65d36ac837fada387efead: Status 404 returned error can't find the container with id 7ae26d960ded9cc953c95f6395626d4c4173bf0dff65d36ac837fada387efead Jul 16 04:38:59.071403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.071375 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:38:59.076320 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.076300 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod053d5cd9_8f78_4549_bdcc_5139d8b44bd1.slice/crio-a4f0a26c5fa0f9174c654e9e3eaec69aa33b2d7e0a9baaa5dfba017edef58e17 WatchSource:0}: Error finding container a4f0a26c5fa0f9174c654e9e3eaec69aa33b2d7e0a9baaa5dfba017edef58e17: Status 404 returned error can't find the container with id a4f0a26c5fa0f9174c654e9e3eaec69aa33b2d7e0a9baaa5dfba017edef58e17 Jul 16 04:38:59.080501 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:38:59.080479 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32e28d9_9cde_4522_af4b_bac6b04ab850.slice/crio-3f4c0f3136d44a2c640f8ec40a7d5c8fb668d9608c71b2b52eb7bb3fdd6bc7e4 WatchSource:0}: Error finding container 3f4c0f3136d44a2c640f8ec40a7d5c8fb668d9608c71b2b52eb7bb3fdd6bc7e4: Status 404 returned error can't find the container with id 3f4c0f3136d44a2c640f8ec40a7d5c8fb668d9608c71b2b52eb7bb3fdd6bc7e4 Jul 16 04:38:59.323069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.322983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:38:59.323069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.323018 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323135 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323154 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323167 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323174 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323257 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:39:00.323235244 +0000 UTC m=+4.049605525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:38:59.323321 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.323295 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:39:00.32327052 +0000 UTC m=+4.049640823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:38:59.423458 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.423419 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:38:59.423625 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.423563 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:38:59.423704 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:38:59.423644 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:00.423625488 +0000 UTC m=+4.149995789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:38:59.525029 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.524954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"3f4c0f3136d44a2c640f8ec40a7d5c8fb668d9608c71b2b52eb7bb3fdd6bc7e4"} Jul 16 04:38:59.529693 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.529603 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" event={"ID":"945aa51d-2a67-49f2-b40e-c4829c655f85","Type":"ContainerStarted","Data":"7ae26d960ded9cc953c95f6395626d4c4173bf0dff65d36ac837fada387efead"} Jul 16 04:38:59.531542 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.531476 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jwqzp" event={"ID":"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5","Type":"ContainerStarted","Data":"21af05ca60e25654afd523722f69c97a12de83a6b66e10bbc34fd5331db0d668"} Jul 16 04:38:59.535805 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.535747 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tq4p6" event={"ID":"dd48f9f6-380b-4ab0-b5b1-02341b0d9801","Type":"ContainerStarted","Data":"4ac64ba89968c0fd82a1023152fc2c2e5fc07df4c06d7d862cb4149bd7487f46"} Jul 16 04:38:59.539437 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.539409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zpjzr" event={"ID":"db73ec87-38ea-4629-a015-bdb64c988d73","Type":"ContainerStarted","Data":"bc07a3f7ab1d7ce581b16e65c007dc9d8988ca85eb276d589a1fe91b2a7a7416"} Jul 16 04:38:59.545099 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.545072 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerStarted","Data":"7e6c0f8486cb3b9d1ed907baef196fa2ce0ddbf7ce8d1b70c09ef4c474582c5d"} Jul 16 04:38:59.548031 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.548005 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgbzl" event={"ID":"053d5cd9-8f78-4549-bdcc-5139d8b44bd1","Type":"ContainerStarted","Data":"a4f0a26c5fa0f9174c654e9e3eaec69aa33b2d7e0a9baaa5dfba017edef58e17"} Jul 16 04:38:59.553108 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.553080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7rqwt" event={"ID":"c3a26870-a25b-474e-a245-ba01fcd77b05","Type":"ContainerStarted","Data":"d186cdcb79eff9f8d7a3c9ccbad7ddcec06aff1026039d27fa647805ba9ed27e"} Jul 16 04:38:59.556414 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.556389 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" event={"ID":"e6d40b44-4d07-47f8-9804-d474de7ee666","Type":"ContainerStarted","Data":"db559e1da4378cfbe11534a239f934d1dd2e8cc2d65d0b63e518991c6108e815"} Jul 16 04:38:59.561338 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.561309 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" event={"ID":"c4a62ec9b3b57c8a82085d965555b5bf","Type":"ContainerStarted","Data":"c31d3f2ac95bbc808600fea8348b4d2b1b50b905d9801f1afef8ed176bd01494"} Jul 16 04:38:59.564934 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.564759 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"efb47399fcbe75804be2359f043ba9c0","Type":"ContainerStarted","Data":"dbf5b0d7b0606a59cda2a6adf485e3307ba00dee9a8b5dfb7e17a713aee1863b"} Jul 16 04:38:59.869642 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.869590 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:33:59 +0000 UTC" deadline="2028-01-13 10:00:49.869615885 +0000 UTC" Jul 16 04:38:59.869642 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:38:59.869638 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13109h21m49.999982708s" Jul 16 04:39:00.334865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.334762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:00.334865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.334811 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:00.335115 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.334974 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:00.335115 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.334998 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:00.335115 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.335013 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:00.335115 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.335076 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:39:02.335057464 +0000 UTC m=+6.061427757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:00.335401 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.335308 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:00.335401 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.335358 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:39:02.33534142 +0000 UTC m=+6.061711702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:00.436242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.436190 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:00.436463 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.436344 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:00.436463 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.436427 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:02.436404112 +0000 UTC m=+6.162774432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:00.507308 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.507266 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:00.507510 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.507422 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:00.507928 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.507901 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:00.508046 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.508030 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:00.508119 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.508104 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:00.508238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:00.508208 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:00.870732 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.870680 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 04:33:59 +0000 UTC" deadline="2028-04-30 17:12:38.34180903 +0000 UTC" Jul 16 04:39:00.870732 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:00.870728 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15708h33m37.471085948s" Jul 16 04:39:02.357346 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.357064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:02.357873 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.357377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:02.357873 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.357292 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:02.357873 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.357665 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:39:06.357641775 +0000 UTC m=+10.084012082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:02.358054 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.357583 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:02.358113 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.358063 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:02.358113 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.358079 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:02.358263 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.358130 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:39:06.358114736 +0000 UTC m=+10.084485031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:02.458283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.458241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:02.458479 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.458410 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:02.458546 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.458488 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:06.458467791 +0000 UTC m=+10.184838092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.507967 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.507976 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.508118 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.508113 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.508237 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:02.508519 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:02.508343 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:02.579019 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:02.578221 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"efb47399fcbe75804be2359f043ba9c0","Type":"ContainerStarted","Data":"ec28a0228b182fb81b90858e2bce599c1561c2236b06bb1e2ba943dc251bc886"} Jul 16 04:39:04.507758 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:04.507712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:04.508255 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:04.507712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:04.508255 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:04.507730 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:04.508255 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:04.507850 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:04.508255 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:04.507987 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:04.508255 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:04.508057 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:04.585063 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:04.584992 2573 generic.go:356] "Generic (PLEG): container finished" podID="efb47399fcbe75804be2359f043ba9c0" containerID="ec28a0228b182fb81b90858e2bce599c1561c2236b06bb1e2ba943dc251bc886" exitCode=0 Jul 16 04:39:04.585257 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:04.585084 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"efb47399fcbe75804be2359f043ba9c0","Type":"ContainerDied","Data":"ec28a0228b182fb81b90858e2bce599c1561c2236b06bb1e2ba943dc251bc886"} Jul 16 04:39:06.391762 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.391716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:06.392273 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.391774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:06.392273 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.391975 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:06.392273 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.391996 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:06.392273 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.392009 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:06.392273 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.392071 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:39:14.392051392 +0000 UTC m=+18.118421693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:06.392545 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.392285 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:06.392545 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.392328 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:39:14.392314303 +0000 UTC m=+18.118684580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:06.493120 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.493036 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:06.493312 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.493191 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:06.493312 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.493261 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:14.493243342 +0000 UTC m=+18.219613624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.507888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.507938 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.508016 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:06.508080 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.508217 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:06.508358 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:06.508300 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:08.507577 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:08.507540 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:08.507577 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:08.507569 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:08.508090 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:08.507696 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:08.508090 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:08.507732 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:08.508090 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:08.507827 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:08.508090 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:08.507914 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:10.507519 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:10.507478 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:10.507990 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:10.507529 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:10.507990 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:10.507529 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:10.507990 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:10.507623 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:10.507990 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:10.507713 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:10.507990 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:10.507766 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:12.507899 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:12.507862 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:12.507899 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:12.507905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:12.508432 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:12.507968 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:12.508432 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:12.507983 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:12.508432 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:12.508041 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:12.508432 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:12.508121 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:14.447639 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.447595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:14.447639 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.447642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447763 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447786 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447804 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447816 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447840 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:39:30.447806113 +0000 UTC m=+34.174176404 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:14.448238 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.447862 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:39:30.447847739 +0000 UTC m=+34.174218042 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:14.507673 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.507638 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:14.507673 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.507670 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:14.507917 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.507768 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:14.507917 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.507838 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:14.507917 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.507852 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:14.508097 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.507964 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:14.548606 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:14.548573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:14.548766 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.548693 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:14.548766 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:14.548743 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:30.548728107 +0000 UTC m=+34.275098397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:16.507809 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:16.507771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:16.508240 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:16.507771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:16.508240 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:16.507914 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:16.508240 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:16.507771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:16.508240 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:16.507966 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:16.508240 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:16.508061 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:18.507310 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:18.507277 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:18.507776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:18.507277 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:18.507776 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:18.507398 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:18.507776 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:18.507505 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:18.507776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:18.507278 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:18.507776 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:18.507599 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:19.624894 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:19.624489 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" event={"ID":"efb47399fcbe75804be2359f043ba9c0","Type":"ContainerStarted","Data":"3fb97adc2642e9075e218e37b871b466deea322cc306be267a4f65ea0568416b"} Jul 16 04:39:19.642530 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:19.642337 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" event={"ID":"c4a62ec9b3b57c8a82085d965555b5bf","Type":"ContainerStarted","Data":"8da5cc0fdb25aec0817cbade853d1580ee4f66cc4b4574305ae5ad647e2207dd"} Jul 16 04:39:19.650562 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:19.650463 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-187.ec2.internal" podStartSLOduration=21.650442199 podStartE2EDuration="21.650442199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:19.649972899 +0000 UTC m=+23.376343200" watchObservedRunningTime="2026-07-16 04:39:19.650442199 +0000 UTC m=+23.376812499" Jul 16 04:39:19.682523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:19.682434 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" podStartSLOduration=2.329089503 podStartE2EDuration="22.682411665s" totalImagesPullingTime="20.353322162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:19.6823727 +0000 UTC m=+23.408743000" watchObservedRunningTime="2026-07-16 04:39:19.682411665 +0000 UTC m=+23.408781965" Jul 16 04:39:19.704908 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:19.704864 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-187.ec2.internal" podStartSLOduration=21.704847647 podStartE2EDuration="21.704847647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:19.704289743 +0000 UTC m=+23.430660042" watchObservedRunningTime="2026-07-16 04:39:19.704847647 +0000 UTC m=+23.431217945" Jul 16 04:39:20.507780 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.507556 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:20.508069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.507605 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:20.508069 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:20.507884 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:20.508069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.507611 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:20.508069 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:20.507996 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:20.508069 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:20.508047 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:20.647833 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.647811 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648165 2573 generic.go:356] "Generic (PLEG): container finished" podID="d32e28d9-9cde-4522-af4b-bac6b04ab850" containerID="105f1f35c5d252ae2a1a3406c8fe35127c25c5cd6c8a88b00935cd4665fa05b0" exitCode=1 Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648242 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"0c6b679bb40d11bf459dc6cc65a0241f1e7670a5161abaeb6ca0c3cb393ea066"} Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648319 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"277f43ade1685fccdbca3a6fff470c0c325306ced5b9e39429c3da7b49d3e15f"} Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648356 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"12f2955390e2d1b76b8ee5975467c3ec68e48899e3d8c83f392a7ca3bc4d0c86"} Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648394 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"ebda4bf6eb572ccdbd3ff5db63354c6a661c2adfc67330da2fcedf8d66d1dcbe"} Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648424 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerDied","Data":"105f1f35c5d252ae2a1a3406c8fe35127c25c5cd6c8a88b00935cd4665fa05b0"} Jul 16 04:39:20.648557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.648463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"c08cca3eba9a36266a5389477ee76a17e6f811b80ec3f8127686565a0dd2a7a2"} Jul 16 04:39:20.649719 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.649697 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnfrm" event={"ID":"945aa51d-2a67-49f2-b40e-c4829c655f85","Type":"ContainerStarted","Data":"059f6d0e501cce630878b922dda72025b213eeb10f88ec8e5694cef104b546b4"} Jul 16 04:39:20.651348 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.651325 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jwqzp" event={"ID":"8c038c5b-915b-4ef6-a784-d55fdd3ad0f5","Type":"ContainerStarted","Data":"457de47c0590a49422a889a0ab60b3f92558aa831b557a0f4e0e5cade1cc2a0f"} Jul 16 04:39:20.652744 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.652720 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tq4p6" event={"ID":"dd48f9f6-380b-4ab0-b5b1-02341b0d9801","Type":"ContainerStarted","Data":"4d6117e7b6ab89fce5c49052b86b6e56141eed33fe8fa0042dfc7f4d44f8b772"} Jul 16 04:39:20.654088 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.654068 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zpjzr" event={"ID":"db73ec87-38ea-4629-a015-bdb64c988d73","Type":"ContainerStarted","Data":"9b05c1c40de03947b6e073b64bbd8dedcfb889a8f0c9b47d6f4cf1fa3a8b40e6"} Jul 16 04:39:20.655982 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.655959 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="3629e87ffd0741be7581a5eee94414ee0f73df8d1f398a98853fb19b176e376e" exitCode=0 Jul 16 04:39:20.656074 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.656034 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"3629e87ffd0741be7581a5eee94414ee0f73df8d1f398a98853fb19b176e376e"} Jul 16 04:39:20.657486 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.657462 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgbzl" event={"ID":"053d5cd9-8f78-4549-bdcc-5139d8b44bd1","Type":"ContainerStarted","Data":"bdbc6753fd2fbb67aa8466ccc7bbac563074a0f5892e5336851a4ae4184b5eb2"} Jul 16 04:39:20.658906 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.658881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7rqwt" event={"ID":"c3a26870-a25b-474e-a245-ba01fcd77b05","Type":"ContainerStarted","Data":"96cd7ad82679e9c7c5f1c116f5cf44dcc3474925ab8e988ce3561bdf118b083c"} Jul 16 04:39:20.660160 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.660123 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" event={"ID":"e6d40b44-4d07-47f8-9804-d474de7ee666","Type":"ContainerStarted","Data":"e46f9a8c4b3c558c15926adfc69668069d9faff52702caeebdde83711f7bf393"} Jul 16 04:39:20.685153 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.685088 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jwqzp" podStartSLOduration=3.32918684 podStartE2EDuration="23.685073633s" totalImagesPullingTime="20.355886793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:20.66669373 +0000 UTC m=+24.393064029" watchObservedRunningTime="2026-07-16 04:39:20.685073633 +0000 UTC m=+24.411443932" Jul 16 04:39:20.685427 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.685402 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zpjzr" podStartSLOduration=3.288335011 podStartE2EDuration="23.685394182s" totalImagesPullingTime="20.397059171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:20.684954103 +0000 UTC m=+24.411324399" watchObservedRunningTime="2026-07-16 04:39:20.685394182 +0000 UTC m=+24.411764481" Jul 16 04:39:20.715890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.715838 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tq4p6" podStartSLOduration=3.353502123 podStartE2EDuration="23.715822659s" totalImagesPullingTime="20.362320536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:20.700561848 +0000 UTC m=+24.426932147" watchObservedRunningTime="2026-07-16 04:39:20.715822659 +0000 UTC m=+24.442192958" Jul 16 04:39:20.716060 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.715992 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kgbzl" podStartSLOduration=3.372283968 podStartE2EDuration="23.71598557s" totalImagesPullingTime="20.343701602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:20.715289221 +0000 UTC m=+24.441659519" watchObservedRunningTime="2026-07-16 04:39:20.71598557 +0000 UTC m=+24.442355894" Jul 16 04:39:20.732529 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:20.732470 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7rqwt" podStartSLOduration=3.347975187 podStartE2EDuration="23.732455453s" totalImagesPullingTime="20.384480266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:20.732208999 +0000 UTC m=+24.458579298" watchObservedRunningTime="2026-07-16 04:39:20.732455453 +0000 UTC m=+24.458825752" Jul 16 04:39:21.217020 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:21.216980 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 04:39:21.665632 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:21.665133 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" event={"ID":"e6d40b44-4d07-47f8-9804-d474de7ee666","Type":"ContainerStarted","Data":"50fbfbbeba7440a071beef49de14ab5133a902b3c87ba061ae62dd1b3fae307c"} Jul 16 04:39:21.803830 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:21.803704 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T04:39:21.2170032Z","UUID":"372ffada-53b3-4a3c-b81e-5ebd7e7e82ea","Handler":null,"Name":"","Endpoint":""} Jul 16 04:39:21.805797 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:21.805769 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 04:39:21.805915 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:21.805828 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 04:39:22.508225 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.508002 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:22.508393 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.508005 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:22.508393 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:22.508304 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:22.508393 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:22.508377 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:22.508582 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.508050 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:22.508582 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:22.508471 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:22.671586 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.671559 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:39:22.672066 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.672042 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"01f2d821c241a63679334c8744890a0f951c3deb4bb0ad4cf30bbe9406e5637c"} Jul 16 04:39:22.674371 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:22.674342 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" event={"ID":"e6d40b44-4d07-47f8-9804-d474de7ee666","Type":"ContainerStarted","Data":"ba15a3b5e2e255efe05ee7c877d66affd858df81ad21ff4a6b3b27fa2c9a11d1"} Jul 16 04:39:24.220953 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.220914 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:39:24.221648 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.221589 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:39:24.243524 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.243466 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-67b84" podStartSLOduration=8.908462993 podStartE2EDuration="27.243447118s" totalImagesPullingTime="18.334984125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:22.70688415 +0000 UTC m=+26.433254449" watchObservedRunningTime="2026-07-16 04:39:24.243447118 +0000 UTC m=+27.969817419" Jul 16 04:39:24.508128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.508053 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:24.508128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.508089 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:24.508128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.508102 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:24.508410 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:24.508200 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:24.508410 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:24.508342 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:24.508522 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:24.508442 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:24.678367 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.678337 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:39:24.678848 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:24.678825 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tq4p6" Jul 16 04:39:25.683203 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.683010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:39:25.683860 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.683561 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"bea26099bed4ddfaad44c265c08700caa99659a598a81665333daaff0373873b"} Jul 16 04:39:25.683860 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.683817 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:39:25.684029 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.684013 2573 scope.go:117] "RemoveContainer" containerID="105f1f35c5d252ae2a1a3406c8fe35127c25c5cd6c8a88b00935cd4665fa05b0" Jul 16 04:39:25.685559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.685538 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="252e27614968d14fddcc35750599221ecd149443a4c7e40d1914e7f4dec3e43b" exitCode=0 Jul 16 04:39:25.685652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.685594 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"252e27614968d14fddcc35750599221ecd149443a4c7e40d1914e7f4dec3e43b"} Jul 16 04:39:25.700516 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:25.700497 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:39:26.507495 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.507455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:26.507689 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.507560 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:26.507689 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.507589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:26.507689 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.507579 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:26.507830 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.507684 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:26.507830 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.507795 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:26.692138 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.692114 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:39:26.692673 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.692533 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" event={"ID":"d32e28d9-9cde-4522-af4b-bac6b04ab850","Type":"ContainerStarted","Data":"0f4348168aa492205afc79be325d071d2da321484c9ad740c738c6b50110334c"} Jul 16 04:39:26.692673 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.692651 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 04:39:26.693157 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.693121 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:39:26.709520 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.709488 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:39:26.737638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.737606 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:39:26.796191 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.796072 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" podStartSLOduration=9.324657894 podStartE2EDuration="29.796054076s" totalImagesPullingTime="20.471396182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:26.74736443 +0000 UTC m=+30.473734755" watchObservedRunningTime="2026-07-16 04:39:26.796054076 +0000 UTC m=+30.522424375" Jul 16 04:39:26.848164 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.848116 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpthh"] Jul 16 04:39:26.848330 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.848230 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:26.848384 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.848348 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:26.848745 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.848723 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccprx"] Jul 16 04:39:26.848847 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.848796 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:26.848921 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.848897 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:26.849328 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.849308 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pb7f"] Jul 16 04:39:26.849395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:26.849375 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:26.849504 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:26.849485 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:27.699580 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:27.699407 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="b081deb209cefc696c7e7f08916d287030146a989267177715b2289caf1936db" exitCode=0 Jul 16 04:39:27.699994 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:27.699447 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"b081deb209cefc696c7e7f08916d287030146a989267177715b2289caf1936db"} Jul 16 04:39:28.507180 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:28.507136 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:28.507342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:28.507189 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:28.507342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:28.507236 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:28.507342 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:28.507244 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:28.507342 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:28.507315 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:28.507508 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:28.507367 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:29.706914 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:29.706885 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="6f9fd9d0edc07a81421f82a258e3f963e8e1dc2ddf84d6743da13f83e79e0238" exitCode=0 Jul 16 04:39:29.707688 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:29.706931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"6f9fd9d0edc07a81421f82a258e3f963e8e1dc2ddf84d6743da13f83e79e0238"} Jul 16 04:39:30.463443 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.463407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:30.463638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.463454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:30.463638 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463592 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:30.463755 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463679 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:40:02.463656033 +0000 UTC m=+66.190026327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 04:39:30.463755 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463597 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 04:39:30.463755 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463740 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 04:39:30.463913 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463759 2573 projected.go:196] Error preparing data for projected volume kube-api-access-kxtjs for pod openshift-network-diagnostics/network-check-target-4pb7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:30.463913 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.463818 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs podName:37c13b7f-3b7d-4165-9b57-7a4c1850d8db nodeName:}" failed. No retries permitted until 2026-07-16 04:40:02.46379759 +0000 UTC m=+66.190167891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxtjs" (UniqueName: "kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs") pod "network-check-target-4pb7f" (UID: "37c13b7f-3b7d-4165-9b57-7a4c1850d8db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 04:39:30.508192 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.508130 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:30.508192 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.508174 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:30.508431 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.508139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:30.508431 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.508303 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4pb7f" podUID="37c13b7f-3b7d-4165-9b57-7a4c1850d8db" Jul 16 04:39:30.508527 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.508433 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccprx" podUID="03421519-f85c-4e08-aa29-5b1f437f72bf" Jul 16 04:39:30.508588 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.508544 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpthh" podUID="b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79" Jul 16 04:39:30.564712 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:30.564682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:30.564888 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.564846 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:30.564950 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:30.564921 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret podName:b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:02.564905029 +0000 UTC m=+66.291275320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret") pod "global-pull-secret-syncer-vpthh" (UID: "b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79") : object "kube-system"/"original-pull-secret" not registered Jul 16 04:39:32.013528 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.013495 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-187.ec2.internal" event="NodeReady" Jul 16 04:39:32.013974 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.013662 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 04:39:32.084350 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.084318 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:39:32.086249 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.086222 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.090009 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.089987 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 04:39:32.090466 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.090444 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xvsxh\"" Jul 16 04:39:32.091018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.091000 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 04:39:32.092022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.092003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 04:39:32.101872 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.101855 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 04:39:32.110100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.110081 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-57lc4"] Jul 16 04:39:32.111769 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.111750 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.114443 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.114422 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 04:39:32.114544 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.114458 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmfl6\"" Jul 16 04:39:32.114801 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.114783 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 04:39:32.115059 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.115041 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v82sc"] Jul 16 04:39:32.116704 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.116671 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.119127 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.119017 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:39:32.120421 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.119770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 04:39:32.120421 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.119770 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 04:39:32.120814 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.120797 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ggd6f\"" Jul 16 04:39:32.120900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.120880 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 04:39:32.134899 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.134880 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v82sc"] Jul 16 04:39:32.145998 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.145975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57lc4"] Jul 16 04:39:32.179468 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgp7v\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179595 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179816 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179816 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.179816 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.179746 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.280712 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.280712 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280670 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.280712 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5648f2ca-69e3-4286-9af0-ef623a7192bd-config-volume\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280722 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5648f2ca-69e3-4286-9af0-ef623a7192bd-tmp-dir\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgp9l\" (UniqueName: \"kubernetes.io/projected/5648f2ca-69e3-4286-9af0-ef623a7192bd-kube-api-access-fgp9l\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgp7v\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d722f5c-1317-4c46-aee0-ccabafad67a2-cert\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.280977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.280958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281379 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281379 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281475 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281414 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281526 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.281481 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:32.281526 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.281495 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:32.281619 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.281560 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:32.781545834 +0000 UTC m=+36.507916110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:32.281619 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281569 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281718 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42xzb\" (UniqueName: \"kubernetes.io/projected/5d722f5c-1317-4c46-aee0-ccabafad67a2-kube-api-access-42xzb\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.281718 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.281830 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.281795 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.285596 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.285575 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.285596 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.285592 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.298717 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.298524 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgp7v\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.298717 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.298714 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.383017 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.382980 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d722f5c-1317-4c46-aee0-ccabafad67a2-cert\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.383235 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42xzb\" (UniqueName: \"kubernetes.io/projected/5d722f5c-1317-4c46-aee0-ccabafad67a2-kube-api-access-42xzb\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.383235 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383158 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5648f2ca-69e3-4286-9af0-ef623a7192bd-config-volume\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.383235 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5648f2ca-69e3-4286-9af0-ef623a7192bd-tmp-dir\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.383235 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgp9l\" (UniqueName: \"kubernetes.io/projected/5648f2ca-69e3-4286-9af0-ef623a7192bd-kube-api-access-fgp9l\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.383433 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383287 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.383433 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.383422 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:32.383525 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.383498 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:39:32.883476872 +0000 UTC m=+36.609847196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:32.383627 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.383595 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5648f2ca-69e3-4286-9af0-ef623a7192bd-tmp-dir\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.384067 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.384041 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5648f2ca-69e3-4286-9af0-ef623a7192bd-config-volume\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.385817 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.385797 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d722f5c-1317-4c46-aee0-ccabafad67a2-cert\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.405791 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.405770 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgp9l\" (UniqueName: \"kubernetes.io/projected/5648f2ca-69e3-4286-9af0-ef623a7192bd-kube-api-access-fgp9l\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.407200 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.407177 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42xzb\" (UniqueName: \"kubernetes.io/projected/5d722f5c-1317-4c46-aee0-ccabafad67a2-kube-api-access-42xzb\") pod \"ingress-canary-v82sc\" (UID: \"5d722f5c-1317-4c46-aee0-ccabafad67a2\") " pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.429252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.429221 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v82sc" Jul 16 04:39:32.507978 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.507940 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:39:32.507978 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.507976 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:39:32.508227 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.507959 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:39:32.513496 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513473 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:39:32.513936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513746 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:39:32.513936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6n555\"" Jul 16 04:39:32.513936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513860 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:39:32.513936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513877 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:39:32.514181 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.513949 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjh6n\"" Jul 16 04:39:32.786829 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.786794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:32.787025 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.786948 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:32.787025 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.786963 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:32.787025 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.787016 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:33.786998984 +0000 UTC m=+37.513369286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:32.887647 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:32.887611 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:32.887826 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.887757 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:32.887904 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:32.887831 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:39:33.887812375 +0000 UTC m=+37.614182660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:33.777746 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:33.777714 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v82sc"] Jul 16 04:39:33.782751 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:39:33.782710 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d722f5c_1317_4c46_aee0_ccabafad67a2.slice/crio-1f83c5fff9f68192124047f2f7359f741f15411f8de5280f0ca77bc35deb95fc WatchSource:0}: Error finding container 1f83c5fff9f68192124047f2f7359f741f15411f8de5280f0ca77bc35deb95fc: Status 404 returned error can't find the container with id 1f83c5fff9f68192124047f2f7359f741f15411f8de5280f0ca77bc35deb95fc Jul 16 04:39:33.795572 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:33.795549 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:33.795714 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:33.795697 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:33.795771 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:33.795716 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:33.795771 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:33.795768 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:35.79575258 +0000 UTC m=+39.522122856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:33.896406 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:33.896303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:33.896533 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:33.896427 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:33.896533 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:33.896484 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:39:35.896469406 +0000 UTC m=+39.622839701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:34.723626 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:34.723587 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="18a98e93bee5f5634dfd1569ff177d7b49382bfb91612167c97d36c6974b7b6f" exitCode=0 Jul 16 04:39:34.723825 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:34.723674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"18a98e93bee5f5634dfd1569ff177d7b49382bfb91612167c97d36c6974b7b6f"} Jul 16 04:39:34.724991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:34.724910 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v82sc" event={"ID":"5d722f5c-1317-4c46-aee0-ccabafad67a2","Type":"ContainerStarted","Data":"1f83c5fff9f68192124047f2f7359f741f15411f8de5280f0ca77bc35deb95fc"} Jul 16 04:39:35.731710 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:35.731485 2573 generic.go:356] "Generic (PLEG): container finished" podID="ab0b8cd2-0e23-41ac-b183-41b735183d6e" containerID="34f652a5f37185af8bcc2a931e822d41ebc9736a1a5c10fd9be60af34c15ef15" exitCode=0 Jul 16 04:39:35.732164 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:35.731533 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerDied","Data":"34f652a5f37185af8bcc2a931e822d41ebc9736a1a5c10fd9be60af34c15ef15"} Jul 16 04:39:35.812886 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:35.812853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:35.813069 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:35.813028 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:35.813069 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:35.813051 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:35.813207 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:35.813171 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:39.813130464 +0000 UTC m=+43.539500756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:35.913834 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:35.913781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:35.914023 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:35.913924 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:35.914023 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:35.914005 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:39:39.913985517 +0000 UTC m=+43.640355807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:36.737339 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:36.737302 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" event={"ID":"ab0b8cd2-0e23-41ac-b183-41b735183d6e","Type":"ContainerStarted","Data":"262a126cab65f60b029b9485ef327728350106276df42252b9a142a854568c05"} Jul 16 04:39:36.738687 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:36.738664 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v82sc" event={"ID":"5d722f5c-1317-4c46-aee0-ccabafad67a2","Type":"ContainerStarted","Data":"def13b16fff0e304599c48f203fcab4edd29ded70af7eba46ca036d2ff7ded8b"} Jul 16 04:39:36.780370 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:36.780321 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j8gh6" podStartSLOduration=8.76453796 podStartE2EDuration="39.780305096s" totalImagesPullingTime="31.015767136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:36.780134719 +0000 UTC m=+40.506505017" watchObservedRunningTime="2026-07-16 04:39:36.780305096 +0000 UTC m=+40.506675395" Jul 16 04:39:36.804577 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:36.804525 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v82sc" podStartSLOduration=2.523968122 podStartE2EDuration="4.804509458s" totalImagesPullingTime="2.280541336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:39:36.804190558 +0000 UTC m=+40.530560856" watchObservedRunningTime="2026-07-16 04:39:36.804509458 +0000 UTC m=+40.530879759" Jul 16 04:39:39.841435 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:39.841217 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:39.841839 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:39.841374 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:39.841839 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:39.841528 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:39.841839 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:39.841588 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:39:47.841573063 +0000 UTC m=+51.567943340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:39.942822 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:39.942757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:39.942978 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:39.942919 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:39.943019 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:39.942988 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:39:47.94297066 +0000 UTC m=+51.669340937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:47.898494 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:47.898452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:39:47.898958 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:47.898604 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:39:47.898958 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:47.898624 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:39:47.898958 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:47.898692 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:03.898675378 +0000 UTC m=+67.625045659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:39:47.999390 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:47.999353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:39:47.999517 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:47.999495 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:39:47.999563 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:39:47.999557 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:40:03.999541545 +0000 UTC m=+67.725911821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:39:58.718354 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:39:58.718326 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7c4d9" Jul 16 04:40:02.508314 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.508263 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:40:02.508314 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.508318 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:40:02.511677 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.511650 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 04:40:02.511799 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.511781 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 04:40:02.519234 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:02.519215 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 04:40:02.519339 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:02.519274 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs podName:03421519-f85c-4e08-aa29-5b1f437f72bf nodeName:}" failed. No retries permitted until 2026-07-16 04:41:06.519258764 +0000 UTC m=+130.245629040 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs") pod "network-metrics-daemon-ccprx" (UID: "03421519-f85c-4e08-aa29-5b1f437f72bf") : secret "metrics-daemon-secret" not found Jul 16 04:40:02.521425 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.521408 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 04:40:02.533848 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.533819 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxtjs\" (UniqueName: \"kubernetes.io/projected/37c13b7f-3b7d-4165-9b57-7a4c1850d8db-kube-api-access-kxtjs\") pod \"network-check-target-4pb7f\" (UID: \"37c13b7f-3b7d-4165-9b57-7a4c1850d8db\") " pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:40:02.608940 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.608898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:40:02.611551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.611530 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 04:40:02.621803 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.621780 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79-original-pull-secret\") pod \"global-pull-secret-syncer-vpthh\" (UID: \"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79\") " pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:40:02.820652 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.820567 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpthh" Jul 16 04:40:02.836573 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.836552 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rjh6n\"" Jul 16 04:40:02.843216 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.843193 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:40:02.972128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.972096 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpthh"] Jul 16 04:40:02.975430 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:02.975398 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3817b1e_22ed_4fe3_b28b_3b76bfc8bf79.slice/crio-ea84b75d0a4133a80abf13827b58f1b2b28190f6d57b7c71b2cc97f6a13d3c03 WatchSource:0}: Error finding container ea84b75d0a4133a80abf13827b58f1b2b28190f6d57b7c71b2cc97f6a13d3c03: Status 404 returned error can't find the container with id ea84b75d0a4133a80abf13827b58f1b2b28190f6d57b7c71b2cc97f6a13d3c03 Jul 16 04:40:02.992248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:02.992219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4pb7f"] Jul 16 04:40:02.995529 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:02.995505 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c13b7f_3b7d_4165_9b57_7a4c1850d8db.slice/crio-30ff1d661165b6f1dea8d9c67003d012f334cdfafdbd00b1b18b147d65cc6ffc WatchSource:0}: Error finding container 30ff1d661165b6f1dea8d9c67003d012f334cdfafdbd00b1b18b147d65cc6ffc: Status 404 returned error can't find the container with id 30ff1d661165b6f1dea8d9c67003d012f334cdfafdbd00b1b18b147d65cc6ffc Jul 16 04:40:03.803335 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:03.803283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pb7f" event={"ID":"37c13b7f-3b7d-4165-9b57-7a4c1850d8db","Type":"ContainerStarted","Data":"30ff1d661165b6f1dea8d9c67003d012f334cdfafdbd00b1b18b147d65cc6ffc"} Jul 16 04:40:03.804862 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:03.804831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpthh" event={"ID":"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79","Type":"ContainerStarted","Data":"ea84b75d0a4133a80abf13827b58f1b2b28190f6d57b7c71b2cc97f6a13d3c03"} Jul 16 04:40:03.917162 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:03.917110 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:40:03.917343 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:03.917288 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 04:40:03.917343 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:03.917312 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fd64958c-kvp52: secret "image-registry-tls" not found Jul 16 04:40:03.917459 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:03.917371 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls podName:6512206d-f4f9-4808-a142-cfea97696618 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:35.917353939 +0000 UTC m=+99.643724216 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls") pod "image-registry-77fd64958c-kvp52" (UID: "6512206d-f4f9-4808-a142-cfea97696618") : secret "image-registry-tls" not found Jul 16 04:40:04.018109 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:04.018069 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:04.018311 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:04.018235 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 04:40:04.018311 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:04.018306 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls podName:5648f2ca-69e3-4286-9af0-ef623a7192bd nodeName:}" failed. No retries permitted until 2026-07-16 04:40:36.018285686 +0000 UTC m=+99.744655966 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls") pod "dns-default-57lc4" (UID: "5648f2ca-69e3-4286-9af0-ef623a7192bd") : secret "dns-default-metrics-tls" not found Jul 16 04:40:08.818780 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:08.818742 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4pb7f" event={"ID":"37c13b7f-3b7d-4165-9b57-7a4c1850d8db","Type":"ContainerStarted","Data":"6d9e5a70af3bf4db075b0d243f5685b3f24a853fb086f79fa07bb008ad86a56f"} Jul 16 04:40:08.819216 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:08.818838 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:40:08.820189 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:08.820168 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpthh" event={"ID":"b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79","Type":"ContainerStarted","Data":"15a988828268d068abc48927d216c521fce92ce7d9aa4d0ad944810275e5a26f"} Jul 16 04:40:08.843206 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:08.843164 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4pb7f" podStartSLOduration=66.85969063 podStartE2EDuration="1m11.843131904s" totalImagesPullingTime="4.983441274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:08.84268432 +0000 UTC m=+72.569054619" watchObservedRunningTime="2026-07-16 04:40:08.843131904 +0000 UTC m=+72.569502200" Jul 16 04:40:08.872686 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:08.872630 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vpthh" podStartSLOduration=66.857831587 podStartE2EDuration="1m11.872612052s" totalImagesPullingTime="5.014780465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:08.871764816 +0000 UTC m=+72.598135115" watchObservedRunningTime="2026-07-16 04:40:08.872612052 +0000 UTC m=+72.598982351" Jul 16 04:40:11.983305 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.983270 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr"] Jul 16 04:40:11.986093 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.986071 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:11.988931 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.988908 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 04:40:11.989036 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.988912 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xntmv\"" Jul 16 04:40:11.989502 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.989479 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:40:11.989502 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.989495 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 04:40:11.989666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:11.989515 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 04:40:12.013323 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.013298 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr"] Jul 16 04:40:12.080237 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.080200 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bffcv\" (UniqueName: \"kubernetes.io/projected/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-kube-api-access-bffcv\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.080237 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.080245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.080441 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.080296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-config\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.181394 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.181360 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bffcv\" (UniqueName: \"kubernetes.io/projected/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-kube-api-access-bffcv\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.181566 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.181405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.181566 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.181431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-config\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.181895 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.181873 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-config\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.183534 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.183507 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-serving-cert\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.198435 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.198407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bffcv\" (UniqueName: \"kubernetes.io/projected/f84a151f-dc0a-4e8b-80e4-37aeb8c44a83-kube-api-access-bffcv\") pod \"service-ca-operator-7695fc6c8c-qsxrr\" (UID: \"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.294585 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.294500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" Jul 16 04:40:12.440028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.440006 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr"] Jul 16 04:40:12.442435 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:12.442404 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84a151f_dc0a_4e8b_80e4_37aeb8c44a83.slice/crio-d0e47b55f9b46920d60d852504a53253585680249253af92461059a779bc4c27 WatchSource:0}: Error finding container d0e47b55f9b46920d60d852504a53253585680249253af92461059a779bc4c27: Status 404 returned error can't find the container with id d0e47b55f9b46920d60d852504a53253585680249253af92461059a779bc4c27 Jul 16 04:40:12.830752 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:12.830720 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" event={"ID":"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83","Type":"ContainerStarted","Data":"d0e47b55f9b46920d60d852504a53253585680249253af92461059a779bc4c27"} Jul 16 04:40:14.837884 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:14.837849 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" event={"ID":"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83","Type":"ContainerStarted","Data":"f04d782d96743fd967224f0437ea1d2604bc7d3d4db3a376130f97608aeebb68"} Jul 16 04:40:14.872028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:14.871981 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" podStartSLOduration=2.092407756 podStartE2EDuration="3.87196424s" totalImagesPullingTime="1.779556484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:14.871414877 +0000 UTC m=+78.597785179" watchObservedRunningTime="2026-07-16 04:40:14.87196424 +0000 UTC m=+78.598334539" Jul 16 04:40:16.975695 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.975657 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jh6mv"] Jul 16 04:40:16.978761 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.978743 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:16.984918 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.984889 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 04:40:16.985020 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.984947 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 04:40:16.985020 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.984999 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 04:40:16.985627 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.985611 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-68dzs\"" Jul 16 04:40:16.985705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:16.985651 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 04:40:17.020863 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.020835 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jh6mv"] Jul 16 04:40:17.121110 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.121077 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frts2\" (UniqueName: \"kubernetes.io/projected/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-api-access-frts2\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.121327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.121139 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-data-volume\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.121327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.121229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-crio-socket\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.121327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.121276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.121454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.121371 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.221865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.221826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-data-volume\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222050 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.221919 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-crio-socket\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222050 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.221947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222050 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.221984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222050 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.222025 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frts2\" (UniqueName: \"kubernetes.io/projected/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-api-access-frts2\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222247 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.222068 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:17.222247 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.222070 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-crio-socket\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.222247 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.222183 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.222247 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.222185 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:17.722162731 +0000 UTC m=+81.448533022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : configmap "kube-rbac-proxy" not found Jul 16 04:40:17.222247 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.222243 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:17.722231922 +0000 UTC m=+81.448602212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.222407 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.222247 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-data-volume\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.249182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.249100 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frts2\" (UniqueName: \"kubernetes.io/projected/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-api-access-frts2\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.725100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.725068 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.725100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:17.725106 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:17.725336 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.725207 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:17.725336 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.725225 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:17.725336 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.725269 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:18.725254851 +0000 UTC m=+82.451625127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : configmap "kube-rbac-proxy" not found Jul 16 04:40:17.725336 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:17.725283 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:18.725276505 +0000 UTC m=+82.451646781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:18.548732 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.548701 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz"] Jul 16 04:40:18.551470 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.551452 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" Jul 16 04:40:18.554247 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.554223 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 16 04:40:18.554385 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.554278 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 16 04:40:18.555751 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.555729 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hsgk7\"" Jul 16 04:40:18.578331 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.578309 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz"] Jul 16 04:40:18.631464 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.631431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2g5c\" (UniqueName: \"kubernetes.io/projected/b2868b05-91d9-423c-8ffc-c2e9ff73734a-kube-api-access-b2g5c\") pod \"migrator-7f59b54f6c-gsxmz\" (UID: \"b2868b05-91d9-423c-8ffc-c2e9ff73734a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" Jul 16 04:40:18.702785 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.702754 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7rqwt_c3a26870-a25b-474e-a245-ba01fcd77b05/dns-node-resolver/0.log" Jul 16 04:40:18.732182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.732132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:18.732182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.732185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2g5c\" (UniqueName: \"kubernetes.io/projected/b2868b05-91d9-423c-8ffc-c2e9ff73734a-kube-api-access-b2g5c\") pod \"migrator-7f59b54f6c-gsxmz\" (UID: \"b2868b05-91d9-423c-8ffc-c2e9ff73734a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" Jul 16 04:40:18.732468 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:18.732268 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:18.732468 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.732302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:18.732468 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:18.732340 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.732320866 +0000 UTC m=+84.458691166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : configmap "kube-rbac-proxy" not found Jul 16 04:40:18.732468 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:18.732375 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:18.732468 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:18.732423 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:20.732409573 +0000 UTC m=+84.458779872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:18.743441 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.743417 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2g5c\" (UniqueName: \"kubernetes.io/projected/b2868b05-91d9-423c-8ffc-c2e9ff73734a-kube-api-access-b2g5c\") pod \"migrator-7f59b54f6c-gsxmz\" (UID: \"b2868b05-91d9-423c-8ffc-c2e9ff73734a\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" Jul 16 04:40:18.860258 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.860232 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" Jul 16 04:40:18.999789 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:18.999757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz"] Jul 16 04:40:19.003000 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:19.002969 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2868b05_91d9_423c_8ffc_c2e9ff73734a.slice/crio-32edcbc9c7334f888be1298fc27ae70cdc4b3195c148ce080d91f89fce0fdead WatchSource:0}: Error finding container 32edcbc9c7334f888be1298fc27ae70cdc4b3195c148ce080d91f89fce0fdead: Status 404 returned error can't find the container with id 32edcbc9c7334f888be1298fc27ae70cdc4b3195c148ce080d91f89fce0fdead Jul 16 04:40:19.853432 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:19.853387 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" event={"ID":"b2868b05-91d9-423c-8ffc-c2e9ff73734a","Type":"ContainerStarted","Data":"32edcbc9c7334f888be1298fc27ae70cdc4b3195c148ce080d91f89fce0fdead"} Jul 16 04:40:20.101480 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.101451 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgbzl_053d5cd9-8f78-4549-bdcc-5139d8b44bd1/node-ca/0.log" Jul 16 04:40:20.749497 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.749461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:20.749700 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.749511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:20.749700 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:20.749595 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:20.749700 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:20.749637 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:20.749700 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:20.749688 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:24.749671086 +0000 UTC m=+88.476041380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : configmap "kube-rbac-proxy" not found Jul 16 04:40:20.749847 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:20.749704 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:24.749697865 +0000 UTC m=+88.476068141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:20.857860 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.857825 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" event={"ID":"b2868b05-91d9-423c-8ffc-c2e9ff73734a","Type":"ContainerStarted","Data":"61247f25612c055ad16c350435b4cdf598534372154cf6be68d4b86a9da13fa6"} Jul 16 04:40:20.858341 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.857876 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" event={"ID":"b2868b05-91d9-423c-8ffc-c2e9ff73734a","Type":"ContainerStarted","Data":"b7662ac38320606dd62e19cd131a48d51e398e54bea42b8efd74cf38fba7ee37"} Jul 16 04:40:20.903099 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:20.903060 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v82sc_5d722f5c-1317-4c46-aee0-ccabafad67a2/serve-healthcheck-canary/0.log" Jul 16 04:40:24.782979 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:24.782933 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:24.783418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:24.782990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:24.783418 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:24.783061 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 04:40:24.783418 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:24.783196 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:32.783176027 +0000 UTC m=+96.509546309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : configmap "kube-rbac-proxy" not found Jul 16 04:40:24.783418 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:24.783200 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 04:40:24.783418 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:24.783252 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls podName:d12b3c0d-7c5f-44fb-a91c-a1cdff705271 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:32.783240421 +0000 UTC m=+96.509610700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jh6mv" (UID: "d12b3c0d-7c5f-44fb-a91c-a1cdff705271") : secret "insights-runtime-extractor-tls" not found Jul 16 04:40:32.845396 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:32.845349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:32.845396 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:32.845401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:32.846011 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:32.845984 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:32.847651 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:32.847634 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d12b3c0d-7c5f-44fb-a91c-a1cdff705271-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jh6mv\" (UID: \"d12b3c0d-7c5f-44fb-a91c-a1cdff705271\") " pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:32.887548 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:32.887510 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jh6mv" Jul 16 04:40:33.031753 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:33.031683 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-gsxmz" podStartSLOduration=13.906202121 podStartE2EDuration="15.031656922s" totalImagesPullingTime="1.125454801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:20.880624186 +0000 UTC m=+84.606994484" watchObservedRunningTime="2026-07-16 04:40:33.031656922 +0000 UTC m=+96.758027216" Jul 16 04:40:33.032073 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:33.032053 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jh6mv"] Jul 16 04:40:33.035695 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:33.035670 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12b3c0d_7c5f_44fb_a91c_a1cdff705271.slice/crio-271b12edc81b5f157e6ea619ac0e51d801b63219d70e56b8574c53e547990da4 WatchSource:0}: Error finding container 271b12edc81b5f157e6ea619ac0e51d801b63219d70e56b8574c53e547990da4: Status 404 returned error can't find the container with id 271b12edc81b5f157e6ea619ac0e51d801b63219d70e56b8574c53e547990da4 Jul 16 04:40:33.897724 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:33.897678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jh6mv" event={"ID":"d12b3c0d-7c5f-44fb-a91c-a1cdff705271","Type":"ContainerStarted","Data":"271b12edc81b5f157e6ea619ac0e51d801b63219d70e56b8574c53e547990da4"} Jul 16 04:40:35.971571 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:35.971531 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:40:35.974315 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:35.974288 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"image-registry-77fd64958c-kvp52\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:40:36.006436 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.006407 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xvsxh\"" Jul 16 04:40:36.007353 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.007317 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:40:36.072671 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.072630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:36.075211 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.075184 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5648f2ca-69e3-4286-9af0-ef623a7192bd-metrics-tls\") pod \"dns-default-57lc4\" (UID: \"5648f2ca-69e3-4286-9af0-ef623a7192bd\") " pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:36.328709 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.328631 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmfl6\"" Jul 16 04:40:36.333509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:36.333487 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:37.878219 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:37.878083 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:40:37.881280 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:37.881258 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6512206d_f4f9_4808_a142_cfea97696618.slice/crio-5e7697cbfb9935f411f35825425efaa5b94edcc05f01af29344c49a9d4b9be22 WatchSource:0}: Error finding container 5e7697cbfb9935f411f35825425efaa5b94edcc05f01af29344c49a9d4b9be22: Status 404 returned error can't find the container with id 5e7697cbfb9935f411f35825425efaa5b94edcc05f01af29344c49a9d4b9be22 Jul 16 04:40:37.891808 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:37.891781 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57lc4"] Jul 16 04:40:37.895860 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:37.895828 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5648f2ca_69e3_4286_9af0_ef623a7192bd.slice/crio-4855cf308850d1cc788b5536e44006155e068b8f567b01194a4b62004764cbe5 WatchSource:0}: Error finding container 4855cf308850d1cc788b5536e44006155e068b8f567b01194a4b62004764cbe5: Status 404 returned error can't find the container with id 4855cf308850d1cc788b5536e44006155e068b8f567b01194a4b62004764cbe5 Jul 16 04:40:37.913334 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:37.913305 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jh6mv" event={"ID":"d12b3c0d-7c5f-44fb-a91c-a1cdff705271","Type":"ContainerStarted","Data":"c26b24bad043f6a850924561e4240e1819f3310fca2652b3d0209d82692d83d8"} Jul 16 04:40:37.914452 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:37.914432 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57lc4" event={"ID":"5648f2ca-69e3-4286-9af0-ef623a7192bd","Type":"ContainerStarted","Data":"4855cf308850d1cc788b5536e44006155e068b8f567b01194a4b62004764cbe5"} Jul 16 04:40:37.915493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:37.915459 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" event={"ID":"6512206d-f4f9-4808-a142-cfea97696618","Type":"ContainerStarted","Data":"5e7697cbfb9935f411f35825425efaa5b94edcc05f01af29344c49a9d4b9be22"} Jul 16 04:40:38.921111 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:38.920947 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" event={"ID":"6512206d-f4f9-4808-a142-cfea97696618","Type":"ContainerStarted","Data":"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f"} Jul 16 04:40:38.921111 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:38.921080 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:40:38.977705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:38.977645 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" podStartSLOduration=101.977605978 podStartE2EDuration="1m41.977605978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:38.976028511 +0000 UTC m=+102.702398840" watchObservedRunningTime="2026-07-16 04:40:38.977605978 +0000 UTC m=+102.703976279" Jul 16 04:40:39.825758 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:39.825726 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4pb7f" Jul 16 04:40:42.826711 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.826618 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz"] Jul 16 04:40:42.829523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.829504 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:42.834982 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.834953 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 16 04:40:42.834982 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.834953 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 16 04:40:42.835182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.834953 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 16 04:40:42.835182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.834953 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 16 04:40:42.865273 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.865246 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz"] Jul 16 04:40:42.929500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.929466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71bc3685-3264-460b-a769-4c7ace39a502-tmp\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:42.929500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.929501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/71bc3685-3264-460b-a769-4c7ace39a502-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:42.929736 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.929518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vftnh\" (UniqueName: \"kubernetes.io/projected/71bc3685-3264-460b-a769-4c7ace39a502-kube-api-access-vftnh\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:42.935764 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.935735 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jh6mv" event={"ID":"d12b3c0d-7c5f-44fb-a91c-a1cdff705271","Type":"ContainerStarted","Data":"a3dedea2c6458cc45cd0dcb86b196452ae9e65a2f2ee509d160c0a8fb7eab655"} Jul 16 04:40:42.937486 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.937461 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57lc4" event={"ID":"5648f2ca-69e3-4286-9af0-ef623a7192bd","Type":"ContainerStarted","Data":"0f1e41bef644e773cf08fef95d50528d4c7b2f4b8c23b6cc022e1f6095f7f82e"} Jul 16 04:40:42.937604 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.937503 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57lc4" event={"ID":"5648f2ca-69e3-4286-9af0-ef623a7192bd","Type":"ContainerStarted","Data":"dce9479ba35846d00a2efa83a4be618fbafecff03b163d2de66f9633924b9e3e"} Jul 16 04:40:42.937671 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.937617 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:42.981500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:42.981451 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-57lc4" podStartSLOduration=66.4272566 podStartE2EDuration="1m10.98143357s" totalImagesPullingTime="4.55417697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:42.981067168 +0000 UTC m=+106.707437467" watchObservedRunningTime="2026-07-16 04:40:42.98143357 +0000 UTC m=+106.707803867" Jul 16 04:40:43.030789 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.030755 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71bc3685-3264-460b-a769-4c7ace39a502-tmp\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.030990 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.030802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/71bc3685-3264-460b-a769-4c7ace39a502-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.030990 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.030949 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vftnh\" (UniqueName: \"kubernetes.io/projected/71bc3685-3264-460b-a769-4c7ace39a502-kube-api-access-vftnh\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.031260 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.031237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71bc3685-3264-460b-a769-4c7ace39a502-tmp\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.033293 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.033277 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/71bc3685-3264-460b-a769-4c7ace39a502-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.088261 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.088192 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vftnh\" (UniqueName: \"kubernetes.io/projected/71bc3685-3264-460b-a769-4c7ace39a502-kube-api-access-vftnh\") pod \"klusterlet-addon-workmgr-74df4c95f-2p2hz\" (UID: \"71bc3685-3264-460b-a769-4c7ace39a502\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.138287 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.138251 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:43.316173 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.316124 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz"] Jul 16 04:40:43.319491 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:43.319459 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71bc3685_3264_460b_a769_4c7ace39a502.slice/crio-4d281dfea1c47eca0bdb086d21d94fdc95a4cb9f71e282757cd7c49c41f19110 WatchSource:0}: Error finding container 4d281dfea1c47eca0bdb086d21d94fdc95a4cb9f71e282757cd7c49c41f19110: Status 404 returned error can't find the container with id 4d281dfea1c47eca0bdb086d21d94fdc95a4cb9f71e282757cd7c49c41f19110 Jul 16 04:40:43.942769 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:43.942731 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" event={"ID":"71bc3685-3264-460b-a769-4c7ace39a502","Type":"ContainerStarted","Data":"4d281dfea1c47eca0bdb086d21d94fdc95a4cb9f71e282757cd7c49c41f19110"} Jul 16 04:40:46.762294 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.762256 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5"] Jul 16 04:40:46.770757 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.770732 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:46.779907 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.775654 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 16 04:40:46.779907 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.775899 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-btn2k\"" Jul 16 04:40:46.782483 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.782404 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5"] Jul 16 04:40:46.862045 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.861989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mjwp5\" (UID: \"e0bbf9e5-927b-4455-8a0c-c400b5db2b30\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:46.963431 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:46.963393 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mjwp5\" (UID: \"e0bbf9e5-927b-4455-8a0c-c400b5db2b30\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:46.963610 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:46.963524 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 16 04:40:46.963610 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:40:46.963586 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates podName:e0bbf9e5-927b-4455-8a0c-c400b5db2b30 nodeName:}" failed. No retries permitted until 2026-07-16 04:40:47.463568282 +0000 UTC m=+111.189938562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-mjwp5" (UID: "e0bbf9e5-927b-4455-8a0c-c400b5db2b30") : secret "prometheus-operator-admission-webhook-tls" not found Jul 16 04:40:47.467698 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.467655 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mjwp5\" (UID: \"e0bbf9e5-927b-4455-8a0c-c400b5db2b30\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:47.470365 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.470337 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e0bbf9e5-927b-4455-8a0c-c400b5db2b30-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-mjwp5\" (UID: \"e0bbf9e5-927b-4455-8a0c-c400b5db2b30\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:47.684217 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.684182 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:47.979446 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.979414 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:40:47.984210 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.984188 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:47.993350 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.993295 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 04:40:47.994308 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994255 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 04:40:47.994308 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994274 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 04:40:47.994494 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994356 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 04:40:47.994623 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994599 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 04:40:47.994718 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994651 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 04:40:47.994718 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:47.994669 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 04:40:48.000824 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.000804 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-97p6j\"" Jul 16 04:40:48.016789 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.016766 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:40:48.174409 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174371 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.174409 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174414 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.174656 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.174656 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.174656 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkhq6\" (UniqueName: \"kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.174656 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.174648 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.275912 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.275833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.275912 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.275888 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.275923 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.275957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.276020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkhq6\" (UniqueName: \"kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.276064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276715 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.276687 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.276850 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.276753 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.277345 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.277322 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.278613 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.278590 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.278729 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.278590 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.290087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.290037 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkhq6\" (UniqueName: \"kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6\") pod \"console-fd685fbc5-x2z5z\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.295808 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.295629 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:48.447971 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.447930 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5"] Jul 16 04:40:48.452691 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:48.452659 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bbf9e5_927b_4455_8a0c_c400b5db2b30.slice/crio-2db311c8a96177df4ab9830651a9317b8fa0af768ff52b6db74d1518abbbb1fe WatchSource:0}: Error finding container 2db311c8a96177df4ab9830651a9317b8fa0af768ff52b6db74d1518abbbb1fe: Status 404 returned error can't find the container with id 2db311c8a96177df4ab9830651a9317b8fa0af768ff52b6db74d1518abbbb1fe Jul 16 04:40:48.508304 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.508283 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:40:48.510506 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:48.510481 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b50f09_1334_44e3_9b67_4f9ede41d053.slice/crio-273c456d19ded80b82ea764ad77e72c46764f6aca720010e0e2fe9d992ced327 WatchSource:0}: Error finding container 273c456d19ded80b82ea764ad77e72c46764f6aca720010e0e2fe9d992ced327: Status 404 returned error can't find the container with id 273c456d19ded80b82ea764ad77e72c46764f6aca720010e0e2fe9d992ced327 Jul 16 04:40:48.964010 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.963970 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jh6mv" event={"ID":"d12b3c0d-7c5f-44fb-a91c-a1cdff705271","Type":"ContainerStarted","Data":"ddccaec1281e341fad6c01e0f03862d4e80aafc3a3d825b7f526f66d90923710"} Jul 16 04:40:48.965558 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.965529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" event={"ID":"71bc3685-3264-460b-a769-4c7ace39a502","Type":"ContainerStarted","Data":"4151c8bddfec2baf4a82085819dae9d1bbf2ea2ef68b68d1e55037d4e5e69ef0"} Jul 16 04:40:48.965756 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.965695 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:48.967024 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.966997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" event={"ID":"e0bbf9e5-927b-4455-8a0c-c400b5db2b30","Type":"ContainerStarted","Data":"2db311c8a96177df4ab9830651a9317b8fa0af768ff52b6db74d1518abbbb1fe"} Jul 16 04:40:48.967399 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.967375 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" Jul 16 04:40:48.968177 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:48.968139 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd685fbc5-x2z5z" event={"ID":"a3b50f09-1334-44e3-9b67-4f9ede41d053","Type":"ContainerStarted","Data":"273c456d19ded80b82ea764ad77e72c46764f6aca720010e0e2fe9d992ced327"} Jul 16 04:40:49.039547 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:49.039440 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df4c95f-2p2hz" podStartSLOduration=2.004528712 podStartE2EDuration="7.03941944s" totalImagesPullingTime="5.034890728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:49.035724972 +0000 UTC m=+112.762095270" watchObservedRunningTime="2026-07-16 04:40:49.03941944 +0000 UTC m=+112.765789739" Jul 16 04:40:49.040578 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:49.040532 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jh6mv" podStartSLOduration=17.935436577 podStartE2EDuration="33.040516637s" totalImagesPullingTime="15.10508006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 04:40:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:48.99572614 +0000 UTC m=+112.722096458" watchObservedRunningTime="2026-07-16 04:40:49.040516637 +0000 UTC m=+112.766886935" Jul 16 04:40:49.974019 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:49.973976 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" event={"ID":"e0bbf9e5-927b-4455-8a0c-c400b5db2b30","Type":"ContainerStarted","Data":"43e4bd6e4f26beabac6ad0d86f394da247a68321172571eb3164a1e18684f819"} Jul 16 04:40:49.974452 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:49.974421 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:49.979557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:49.979532 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" Jul 16 04:40:50.002916 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.002865 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-mjwp5" podStartSLOduration=3.096213405 podStartE2EDuration="4.002848072s" totalImagesPullingTime="906.634667ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:50.001257353 +0000 UTC m=+113.727627650" watchObservedRunningTime="2026-07-16 04:40:50.002848072 +0000 UTC m=+113.729218369" Jul 16 04:40:50.178381 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.178345 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:40:50.181495 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.181472 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.257307 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.257203 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 04:40:50.278480 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.278450 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:40:50.295423 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295586 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295440 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295586 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6dgm\" (UniqueName: \"kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295586 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295576 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295750 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295607 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295750 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.295750 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.295709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396295 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6dgm\" (UniqueName: \"kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.396795 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.396767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.397186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.397123 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.397186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.397168 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.397340 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.397169 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.397678 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.397657 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.399441 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.399420 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.399537 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.399487 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.446495 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.446461 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6dgm\" (UniqueName: \"kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm\") pod \"console-cdcc4d4b6-jvd6h\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:50.492207 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:50.492168 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:40:51.411241 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:51.411207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:40:51.411814 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:40:51.411787 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03564964_f6da_4692_8c99_9aabd7c61a20.slice/crio-1a956ce3e3398e67ce20bbba5e73176188f6b85c8de956cd6c9b77f73f596547 WatchSource:0}: Error finding container 1a956ce3e3398e67ce20bbba5e73176188f6b85c8de956cd6c9b77f73f596547: Status 404 returned error can't find the container with id 1a956ce3e3398e67ce20bbba5e73176188f6b85c8de956cd6c9b77f73f596547 Jul 16 04:40:51.983198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:51.983160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdcc4d4b6-jvd6h" event={"ID":"03564964-f6da-4692-8c99-9aabd7c61a20","Type":"ContainerStarted","Data":"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a"} Jul 16 04:40:51.983379 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:51.983214 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdcc4d4b6-jvd6h" event={"ID":"03564964-f6da-4692-8c99-9aabd7c61a20","Type":"ContainerStarted","Data":"1a956ce3e3398e67ce20bbba5e73176188f6b85c8de956cd6c9b77f73f596547"} Jul 16 04:40:51.984557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:51.984533 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd685fbc5-x2z5z" event={"ID":"a3b50f09-1334-44e3-9b67-4f9ede41d053","Type":"ContainerStarted","Data":"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586"} Jul 16 04:40:52.037347 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:52.035634 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cdcc4d4b6-jvd6h" podStartSLOduration=2.035615492 podStartE2EDuration="2.035615492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:52.034112588 +0000 UTC m=+115.760482887" watchObservedRunningTime="2026-07-16 04:40:52.035615492 +0000 UTC m=+115.761985791" Jul 16 04:40:52.075343 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:52.075281 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fd685fbc5-x2z5z" podStartSLOduration=2.250858772 podStartE2EDuration="5.075264673s" totalImagesPullingTime="2.824405901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:40:52.075022811 +0000 UTC m=+115.801393132" watchObservedRunningTime="2026-07-16 04:40:52.075264673 +0000 UTC m=+115.801634970" Jul 16 04:40:52.945195 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:52.945139 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-57lc4" Jul 16 04:40:56.011646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:56.011604 2573 patch_prober.go:28] interesting pod/image-registry-77fd64958c-kvp52 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 04:40:56.012034 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:56.011666 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 04:40:58.296454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:58.296421 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:58.296903 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:58.296483 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:58.301171 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:58.301132 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:59.011407 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:59.011379 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:40:59.929196 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:40:59.929168 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:41:00.492543 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:00.492505 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:00.492723 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:00.492595 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:00.497284 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:00.497258 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:01.018039 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:01.018010 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:01.118443 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:01.118411 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:41:04.791983 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:04.791941 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:41:06.617508 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:06.617469 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:41:06.619786 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:06.619766 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03421519-f85c-4e08-aa29-5b1f437f72bf-metrics-certs\") pod \"network-metrics-daemon-ccprx\" (UID: \"03421519-f85c-4e08-aa29-5b1f437f72bf\") " pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:41:06.731286 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:06.731261 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6n555\"" Jul 16 04:41:06.737166 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:06.737128 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccprx" Jul 16 04:41:06.882333 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:06.882254 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccprx"] Jul 16 04:41:06.885253 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:06.885223 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03421519_f85c_4e08_aa29_5b1f437f72bf.slice/crio-230996b3660e3ef646d713a2d8436bd21565925fca46936c763abd86eb5c5547 WatchSource:0}: Error finding container 230996b3660e3ef646d713a2d8436bd21565925fca46936c763abd86eb5c5547: Status 404 returned error can't find the container with id 230996b3660e3ef646d713a2d8436bd21565925fca46936c763abd86eb5c5547 Jul 16 04:41:07.035988 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.035954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccprx" event={"ID":"03421519-f85c-4e08-aa29-5b1f437f72bf","Type":"ContainerStarted","Data":"230996b3660e3ef646d713a2d8436bd21565925fca46936c763abd86eb5c5547"} Jul 16 04:41:07.056048 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.056012 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8"] Jul 16 04:41:07.061165 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.061134 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ml2rb"] Jul 16 04:41:07.061291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.061261 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.065316 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.065297 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.076794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.076778 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 04:41:07.084712 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.084688 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 04:41:07.084841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.084776 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 04:41:07.084897 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.084776 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 04:41:07.091022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.091003 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8"] Jul 16 04:41:07.094566 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.094549 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 04:41:07.094713 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.094695 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 04:41:07.094900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.094887 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gb5r5\"" Jul 16 04:41:07.094978 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.094894 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 04:41:07.095525 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.095511 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zljch\"" Jul 16 04:41:07.099743 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.099729 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 04:41:07.120688 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfcsp\" (UniqueName: \"kubernetes.io/projected/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-api-access-hfcsp\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.120794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v4v5\" (UniqueName: \"kubernetes.io/projected/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-kube-api-access-9v4v5\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.120794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.120794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120729 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.120794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.120794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.120953 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.120953 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.120953 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.120953 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.120944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3aaabf2f-3945-4ca3-b89c-34017bdd555e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.150630 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.150557 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ml2rb"] Jul 16 04:41:07.151662 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.151637 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s4r7b"] Jul 16 04:41:07.155271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.155254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.161816 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.161798 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 04:41:07.161902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.161826 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kdsbv\"" Jul 16 04:41:07.162060 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.162047 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 04:41:07.162127 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.162101 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 04:41:07.221808 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-wtmp\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.221808 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221811 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-root\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.221989 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222300 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-accelerators-collector-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222300 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222119 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-textfile\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222300 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222203 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.222300 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222242 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222300 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222283 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jsd\" (UniqueName: \"kubernetes.io/projected/b03d00f9-36c7-434c-8fa9-88e00d28ae55-kube-api-access-r9jsd\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222321 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3aaabf2f-3945-4ca3-b89c-34017bdd555e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-tls\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-metrics-client-ca\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfcsp\" (UniqueName: \"kubernetes.io/projected/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-api-access-hfcsp\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222486 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v4v5\" (UniqueName: \"kubernetes.io/projected/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-kube-api-access-9v4v5\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.222546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222510 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-sys\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.222846 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222581 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.222846 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222765 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222961 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222845 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3aaabf2f-3945-4ca3-b89c-34017bdd555e-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.222961 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.222902 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3aaabf2f-3945-4ca3-b89c-34017bdd555e-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.224514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.224489 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.224514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.224501 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.224762 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.224745 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.224923 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.224904 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.238128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.238102 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfcsp\" (UniqueName: \"kubernetes.io/projected/3aaabf2f-3945-4ca3-b89c-34017bdd555e-kube-api-access-hfcsp\") pod \"kube-state-metrics-785778ff74-ml2rb\" (UID: \"3aaabf2f-3945-4ca3-b89c-34017bdd555e\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.238807 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.238787 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v4v5\" (UniqueName: \"kubernetes.io/projected/98839d4e-bdf3-4e2a-9dde-b1b8be065bed-kube-api-access-9v4v5\") pod \"openshift-state-metrics-84bd68b896-l8kg8\" (UID: \"98839d4e-bdf3-4e2a-9dde-b1b8be065bed\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.323111 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jsd\" (UniqueName: \"kubernetes.io/projected/b03d00f9-36c7-434c-8fa9-88e00d28ae55-kube-api-access-r9jsd\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323111 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-tls\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323380 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-metrics-client-ca\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323380 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-sys\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-wtmp\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323437 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-sys\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323443 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-root\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323484 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-root\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323544 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-accelerators-collector-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323560 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-wtmp\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-textfile\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323852 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323828 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-textfile\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.323852 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.323834 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-metrics-client-ca\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.324020 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.324002 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-accelerators-collector-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.325319 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.325300 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-tls\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.325459 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.325444 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b03d00f9-36c7-434c-8fa9-88e00d28ae55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.341415 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.341392 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jsd\" (UniqueName: \"kubernetes.io/projected/b03d00f9-36c7-434c-8fa9-88e00d28ae55-kube-api-access-r9jsd\") pod \"node-exporter-s4r7b\" (UID: \"b03d00f9-36c7-434c-8fa9-88e00d28ae55\") " pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.371194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.371167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" Jul 16 04:41:07.376578 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.376564 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" Jul 16 04:41:07.464566 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.464532 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s4r7b" Jul 16 04:41:07.476844 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:07.476814 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03d00f9_36c7_434c_8fa9_88e00d28ae55.slice/crio-aacde598d7df9124dc9da098ff25d9a7c61d771995da24118f5559d505849ecc WatchSource:0}: Error finding container aacde598d7df9124dc9da098ff25d9a7c61d771995da24118f5559d505849ecc: Status 404 returned error can't find the container with id aacde598d7df9124dc9da098ff25d9a7c61d771995da24118f5559d505849ecc Jul 16 04:41:07.560010 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.559975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8"] Jul 16 04:41:07.563900 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:07.563867 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98839d4e_bdf3_4e2a_9dde_b1b8be065bed.slice/crio-f511639e90b6d4e5585519f4b00c1509803f9161db225417750d85ab0074f3ac WatchSource:0}: Error finding container f511639e90b6d4e5585519f4b00c1509803f9161db225417750d85ab0074f3ac: Status 404 returned error can't find the container with id f511639e90b6d4e5585519f4b00c1509803f9161db225417750d85ab0074f3ac Jul 16 04:41:07.576100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:07.576078 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ml2rb"] Jul 16 04:41:07.579209 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:07.579186 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aaabf2f_3945_4ca3_b89c_34017bdd555e.slice/crio-4cdd37546a823ee6048e3a0dcc8a8bdc4aedd3632fbaaec8b1b106f423c636a7 WatchSource:0}: Error finding container 4cdd37546a823ee6048e3a0dcc8a8bdc4aedd3632fbaaec8b1b106f423c636a7: Status 404 returned error can't find the container with id 4cdd37546a823ee6048e3a0dcc8a8bdc4aedd3632fbaaec8b1b106f423c636a7 Jul 16 04:41:08.040464 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.040390 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" event={"ID":"3aaabf2f-3945-4ca3-b89c-34017bdd555e","Type":"ContainerStarted","Data":"4cdd37546a823ee6048e3a0dcc8a8bdc4aedd3632fbaaec8b1b106f423c636a7"} Jul 16 04:41:08.042088 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.042064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" event={"ID":"98839d4e-bdf3-4e2a-9dde-b1b8be065bed","Type":"ContainerStarted","Data":"9f6eb86be022616045f28266554de3498a4bf1b04d37978a2c3ca5134ef01fbf"} Jul 16 04:41:08.042220 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.042116 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" event={"ID":"98839d4e-bdf3-4e2a-9dde-b1b8be065bed","Type":"ContainerStarted","Data":"546606e6bc7821f6578bd31f9fa20d9dca544f74b1a5d6f23f45fe09138eb18f"} Jul 16 04:41:08.042220 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.042189 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" event={"ID":"98839d4e-bdf3-4e2a-9dde-b1b8be065bed","Type":"ContainerStarted","Data":"f511639e90b6d4e5585519f4b00c1509803f9161db225417750d85ab0074f3ac"} Jul 16 04:41:08.043187 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.043166 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s4r7b" event={"ID":"b03d00f9-36c7-434c-8fa9-88e00d28ae55","Type":"ContainerStarted","Data":"aacde598d7df9124dc9da098ff25d9a7c61d771995da24118f5559d505849ecc"} Jul 16 04:41:08.362413 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.362378 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:41:08.366649 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.366625 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.405860 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:08.405824 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"alertmanager-main-generated\" is forbidden: User \"system:node:ip-10-0-137-187.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-137-187.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" type="*v1.Secret" Jul 16 04:41:08.406182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.406158 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 04:41:08.406264 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.406239 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 04:41:08.406403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.406350 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 04:41:08.406471 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.406456 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 04:41:08.406512 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.406483 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 04:41:08.423157 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.423116 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 04:41:08.423157 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.423134 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 04:41:08.433022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.432993 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 04:41:08.433229 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433337 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433241 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433337 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433436 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433436 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433410 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftxmh\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433475 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433503 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433761 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433694 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433803 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.433803 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.433787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.449116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.449093 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tcwqz\"" Jul 16 04:41:08.535000 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.534963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftxmh\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535106 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535173 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535202 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.535554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.535336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.536771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.536251 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.536771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.536514 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.536771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.536536 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.537920 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.537892 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.538104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.538064 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.538817 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.538789 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.538921 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.538827 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.539623 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.539602 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.539707 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.539685 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.539772 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.539686 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.540100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.540080 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.564552 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.564527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftxmh\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:08.569227 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:08.569199 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:41:09.048963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.048880 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccprx" event={"ID":"03421519-f85c-4e08-aa29-5b1f437f72bf","Type":"ContainerStarted","Data":"42d1a332602d5369dfee7d4357f287dc69a8f7f5d7a704452e7a5062a7532743"} Jul 16 04:41:09.048963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.048952 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccprx" event={"ID":"03421519-f85c-4e08-aa29-5b1f437f72bf","Type":"ContainerStarted","Data":"902b9a64c645a074f8de1a744f625758b7423da794e3873049dd8c0b44db0a8a"} Jul 16 04:41:09.050892 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.050851 2573 generic.go:356] "Generic (PLEG): container finished" podID="b03d00f9-36c7-434c-8fa9-88e00d28ae55" containerID="c8832060e1e565517ddfd5d211fba93e8b1355b46c1c5614e0f6c3160f0b2000" exitCode=0 Jul 16 04:41:09.051000 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.050929 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s4r7b" event={"ID":"b03d00f9-36c7-434c-8fa9-88e00d28ae55","Type":"ContainerDied","Data":"c8832060e1e565517ddfd5d211fba93e8b1355b46c1c5614e0f6c3160f0b2000"} Jul 16 04:41:09.082276 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.082222 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ccprx" podStartSLOduration=130.879291413 podStartE2EDuration="2m12.082200794s" totalImagesPullingTime="1.202909381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:09.081883037 +0000 UTC m=+132.808253339" watchObservedRunningTime="2026-07-16 04:41:09.082200794 +0000 UTC m=+132.808571094" Jul 16 04:41:09.536605 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:09.536565 2573 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-generated: failed to sync secret cache: timed out waiting for the condition Jul 16 04:41:09.536758 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:09.536664 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume podName:c4c318e2-7d54-4acf-ba31-00af99b59d87 nodeName:}" failed. No retries permitted until 2026-07-16 04:41:10.036638792 +0000 UTC m=+133.763009085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-volume" (UniqueName: "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume") pod "alertmanager-main-0" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87") : failed to sync secret cache: timed out waiting for the condition Jul 16 04:41:09.689720 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:09.689605 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 04:41:10.047384 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.047352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:10.050457 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.050430 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") pod \"alertmanager-main-0\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:10.059056 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.058902 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s4r7b" event={"ID":"b03d00f9-36c7-434c-8fa9-88e00d28ae55","Type":"ContainerStarted","Data":"a0308c80552bca39fedef58b7b76688933e395a4b25b7d101c47b39acb51b195"} Jul 16 04:41:10.059056 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.058962 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s4r7b" event={"ID":"b03d00f9-36c7-434c-8fa9-88e00d28ae55","Type":"ContainerStarted","Data":"bad23a27fb941a6c4dad1f94d937308fc6d4bc9ced144f13a9997022b443b91a"} Jul 16 04:41:10.060464 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.060418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" event={"ID":"3aaabf2f-3945-4ca3-b89c-34017bdd555e","Type":"ContainerStarted","Data":"e7fccf96a3f1cee9d38b9aee216892509311ff14f5fc4ffe9050387a02f192fa"} Jul 16 04:41:10.063102 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.063069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" event={"ID":"98839d4e-bdf3-4e2a-9dde-b1b8be065bed","Type":"ContainerStarted","Data":"38eb5f76eeb3c1128b6a61606127ec02196e72755f20b903cae531758e9e7bcb"} Jul 16 04:41:10.165313 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.165268 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s4r7b" podStartSLOduration=1.9461245740000002 podStartE2EDuration="3.165251072s" totalImagesPullingTime="1.156804179s" totalInitContainerRuntime="62.322319ms" isStatefulPod=true podCreationTimestamp="2026-07-16 04:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:10.164051918 +0000 UTC m=+133.890422215" watchObservedRunningTime="2026-07-16 04:41:10.165251072 +0000 UTC m=+133.891621363" Jul 16 04:41:10.177405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.177369 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:41:10.233119 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.233020 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-l8kg8" podStartSLOduration=2.009289118 podStartE2EDuration="4.232996974s" totalImagesPullingTime="2.223707856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:10.232311156 +0000 UTC m=+133.958681482" watchObservedRunningTime="2026-07-16 04:41:10.232996974 +0000 UTC m=+133.959367272" Jul 16 04:41:10.453770 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:10.453743 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:41:10.455557 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:10.455502 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c318e2_7d54_4acf_ba31_00af99b59d87.slice/crio-1f94e6e5a752f38855289c5658e38ff65689c4ec0d8b5b3e4cef789bd8150ac0 WatchSource:0}: Error finding container 1f94e6e5a752f38855289c5658e38ff65689c4ec0d8b5b3e4cef789bd8150ac0: Status 404 returned error can't find the container with id 1f94e6e5a752f38855289c5658e38ff65689c4ec0d8b5b3e4cef789bd8150ac0 Jul 16 04:41:11.069948 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.069905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"1f94e6e5a752f38855289c5658e38ff65689c4ec0d8b5b3e4cef789bd8150ac0"} Jul 16 04:41:11.072347 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.072305 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" event={"ID":"3aaabf2f-3945-4ca3-b89c-34017bdd555e","Type":"ContainerStarted","Data":"174ca088c883c6e0d2998163d71f2f470cc873f20d6b15faf5fe5f5a7e629f89"} Jul 16 04:41:11.072494 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.072367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" event={"ID":"3aaabf2f-3945-4ca3-b89c-34017bdd555e","Type":"ContainerStarted","Data":"a932f5b4b67ec31fc19ce2ef6cf98f090ade73ce5e756618c64a0f8b181b9573"} Jul 16 04:41:11.443329 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.443267 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-ml2rb" podStartSLOduration=2.089851496 podStartE2EDuration="4.443239342s" totalImagesPullingTime="2.353387846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:11.141039199 +0000 UTC m=+134.867409519" watchObservedRunningTime="2026-07-16 04:41:11.443239342 +0000 UTC m=+135.169609640" Jul 16 04:41:11.443849 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.443824 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:11.464033 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.464002 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.487826 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.487757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:11.563834 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.563799 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.563834 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.563841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7psw\" (UniqueName: \"kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.564046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.563869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.564046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.563959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.564046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.563989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.564046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.564014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.564046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.564040 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665473 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665473 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665734 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665493 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665734 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665508 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665734 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7psw\" (UniqueName: \"kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.665890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.665806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.666203 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.666169 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.666330 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.666241 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.666509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.666492 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.666796 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.666774 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.668491 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.668464 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.668683 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.668657 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.686911 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.686881 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7psw\" (UniqueName: \"kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw\") pod \"console-84bb6f4c75-4zn26\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.775247 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.775170 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:11.860544 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.860518 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-688d44c87-fp6dc"] Jul 16 04:41:11.891567 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.890353 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688d44c87-fp6dc"] Jul 16 04:41:11.891567 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.890495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.898693 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.898646 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-35j7erv2jfl95\"" Jul 16 04:41:11.898876 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.898816 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 04:41:11.898955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.898918 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bht74\"" Jul 16 04:41:11.899787 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.899398 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 04:41:11.899787 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.899644 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 04:41:11.900058 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.900034 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 04:41:11.969073 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969199 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-client-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969199 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969115 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d21509a6-dfb9-4919-b473-1c90aa53c908-audit-log\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969280 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-client-certs\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969280 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqvgf\" (UniqueName: \"kubernetes.io/projected/d21509a6-dfb9-4919-b473-1c90aa53c908-kube-api-access-sqvgf\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969356 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-metrics-server-audit-profiles\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.969419 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.969403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-tls\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:11.982031 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:11.982004 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:11.984634 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:11.984612 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abb69dc_e4da_43bd_ae44_ae0e427436d5.slice/crio-d5ac049c7c1638e987a0b75526f37898b0fc36d030a8262534b3127e6c9cb083 WatchSource:0}: Error finding container d5ac049c7c1638e987a0b75526f37898b0fc36d030a8262534b3127e6c9cb083: Status 404 returned error can't find the container with id d5ac049c7c1638e987a0b75526f37898b0fc36d030a8262534b3127e6c9cb083 Jul 16 04:41:12.070865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.070791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-tls\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.070865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.070837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071355 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.070878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-client-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071355 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.070925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d21509a6-dfb9-4919-b473-1c90aa53c908-audit-log\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071355 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.070969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-client-certs\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071355 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.071001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqvgf\" (UniqueName: \"kubernetes.io/projected/d21509a6-dfb9-4919-b473-1c90aa53c908-kube-api-access-sqvgf\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071355 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.071103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-metrics-server-audit-profiles\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071596 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.071360 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d21509a6-dfb9-4919-b473-1c90aa53c908-audit-log\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071853 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.071828 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.071992 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.071974 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d21509a6-dfb9-4919-b473-1c90aa53c908-metrics-server-audit-profiles\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.073271 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.073251 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-client-ca-bundle\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.073431 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.073414 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-tls\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.073485 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.073415 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d21509a6-dfb9-4919-b473-1c90aa53c908-secret-metrics-server-client-certs\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.077938 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.077908 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8" exitCode=0 Jul 16 04:41:12.078039 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.077982 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8"} Jul 16 04:41:12.079647 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.079455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb6f4c75-4zn26" event={"ID":"9abb69dc-e4da-43bd-ae44-ae0e427436d5","Type":"ContainerStarted","Data":"d5ac049c7c1638e987a0b75526f37898b0fc36d030a8262534b3127e6c9cb083"} Jul 16 04:41:12.091138 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.091118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqvgf\" (UniqueName: \"kubernetes.io/projected/d21509a6-dfb9-4919-b473-1c90aa53c908-kube-api-access-sqvgf\") pod \"metrics-server-688d44c87-fp6dc\" (UID: \"d21509a6-dfb9-4919-b473-1c90aa53c908\") " pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.202583 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.202549 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:12.322660 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.322126 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs"] Jul 16 04:41:12.350118 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.350084 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.356944 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.356881 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 16 04:41:12.356944 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.356935 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 16 04:41:12.359387 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.359369 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 16 04:41:12.359703 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.359689 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-nfdlh\"" Jul 16 04:41:12.368452 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:12.368427 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21509a6_dfb9_4919_b473_1c90aa53c908.slice/crio-00e1201ee5d6e7a5b4957dfae384221c799a34cc5d1f9c9140ad9fa01e94cc66 WatchSource:0}: Error finding container 00e1201ee5d6e7a5b4957dfae384221c799a34cc5d1f9c9140ad9fa01e94cc66: Status 404 returned error can't find the container with id 00e1201ee5d6e7a5b4957dfae384221c799a34cc5d1f9c9140ad9fa01e94cc66 Jul 16 04:41:12.372315 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.372300 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 16 04:41:12.373056 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.373039 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 16 04:41:12.374492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.374473 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688d44c87-fp6dc"] Jul 16 04:41:12.475579 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475743 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475591 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475743 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475612 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-metrics-client-ca\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475743 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475708 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-serving-certs-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rssdl\" (UniqueName: \"kubernetes.io/projected/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-kube-api-access-rssdl\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475826 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.475900 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.475852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-federate-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.484994 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.484970 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 16 04:41:12.576780 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.576694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.576780 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.576737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-metrics-client-ca\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.576879 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.576944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-serving-certs-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.576986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rssdl\" (UniqueName: \"kubernetes.io/projected/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-kube-api-access-rssdl\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577176 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.577019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577176 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.577044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-federate-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577176 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.577118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577682 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.577544 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-metrics-client-ca\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.577795 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.577698 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-serving-certs-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.578262 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.578219 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.579814 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.579786 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.580058 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.580041 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-federate-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.580058 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.580053 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-secret-telemeter-client\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.580193 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.580121 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-telemeter-client-tls\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.615659 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.615630 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rssdl\" (UniqueName: \"kubernetes.io/projected/af7efca5-9d0b-498b-afe8-b6ced15f7ac3-kube-api-access-rssdl\") pod \"telemeter-client-7ffd7cf8b5-rr2zs\" (UID: \"af7efca5-9d0b-498b-afe8-b6ced15f7ac3\") " pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.659776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.659753 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" Jul 16 04:41:12.672586 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.672499 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs"] Jul 16 04:41:12.848898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:12.848829 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs"] Jul 16 04:41:12.849249 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:12.849221 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7efca5_9d0b_498b_afe8_b6ced15f7ac3.slice/crio-432177c1fc7e3f4b80de9d0acdac29b647f50b6376c2744aad7d331f1c1eee3a WatchSource:0}: Error finding container 432177c1fc7e3f4b80de9d0acdac29b647f50b6376c2744aad7d331f1c1eee3a: Status 404 returned error can't find the container with id 432177c1fc7e3f4b80de9d0acdac29b647f50b6376c2744aad7d331f1c1eee3a Jul 16 04:41:13.087956 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:13.087922 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb6f4c75-4zn26" event={"ID":"9abb69dc-e4da-43bd-ae44-ae0e427436d5","Type":"ContainerStarted","Data":"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e"} Jul 16 04:41:13.089683 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:13.089633 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" event={"ID":"d21509a6-dfb9-4919-b473-1c90aa53c908","Type":"ContainerStarted","Data":"00e1201ee5d6e7a5b4957dfae384221c799a34cc5d1f9c9140ad9fa01e94cc66"} Jul 16 04:41:13.091067 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:13.091040 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" event={"ID":"af7efca5-9d0b-498b-afe8-b6ced15f7ac3","Type":"ContainerStarted","Data":"432177c1fc7e3f4b80de9d0acdac29b647f50b6376c2744aad7d331f1c1eee3a"} Jul 16 04:41:13.201057 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:13.200997 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84bb6f4c75-4zn26" podStartSLOduration=2.200978152 podStartE2EDuration="2.200978152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:13.198663207 +0000 UTC m=+136.925033509" watchObservedRunningTime="2026-07-16 04:41:13.200978152 +0000 UTC m=+136.927348450" Jul 16 04:41:15.104748 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.104715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" event={"ID":"d21509a6-dfb9-4919-b473-1c90aa53c908","Type":"ContainerStarted","Data":"d4765e74826762a147abbb92f645ac8c6e8cd73fd45f0a602b53aae7e4eba8a2"} Jul 16 04:41:15.108316 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.108295 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b"} Jul 16 04:41:15.108400 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.108349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6"} Jul 16 04:41:15.108400 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.108374 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f"} Jul 16 04:41:15.108400 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.108397 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca"} Jul 16 04:41:15.145813 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:15.145771 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" podStartSLOduration=2.246005882 podStartE2EDuration="4.145754812s" totalImagesPullingTime="1.89974893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:15.143071677 +0000 UTC m=+138.869441986" watchObservedRunningTime="2026-07-16 04:41:15.145754812 +0000 UTC m=+138.872125111" Jul 16 04:41:16.114435 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.114401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" event={"ID":"af7efca5-9d0b-498b-afe8-b6ced15f7ac3","Type":"ContainerStarted","Data":"71974c5fcc866b1590b7f727770eed9bad9d63c03d1549fde7ea129ef9b8da7d"} Jul 16 04:41:16.114884 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.114493 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" event={"ID":"af7efca5-9d0b-498b-afe8-b6ced15f7ac3","Type":"ContainerStarted","Data":"a8dd57f1cc5e87128b01f46a55fa2d58111b6f9161960fc7276c058e00db5359"} Jul 16 04:41:16.114884 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.114525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" event={"ID":"af7efca5-9d0b-498b-afe8-b6ced15f7ac3","Type":"ContainerStarted","Data":"6ae96071ed9c999252a26c6cec9219f1dfd7a5df4bb7e6998f258f917c212c78"} Jul 16 04:41:16.117583 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.117557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9"} Jul 16 04:41:16.117742 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.117637 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerStarted","Data":"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390"} Jul 16 04:41:16.167166 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.149270 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7ffd7cf8b5-rr2zs" podStartSLOduration=1.896606604 podStartE2EDuration="4.149254492s" totalImagesPullingTime="2.252647888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:16.148427162 +0000 UTC m=+139.874797459" watchObservedRunningTime="2026-07-16 04:41:16.149254492 +0000 UTC m=+139.875624789" Jul 16 04:41:16.217868 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:16.217771 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.773088692 podStartE2EDuration="8.21775397s" totalImagesPullingTime="4.419785087s" totalInitContainerRuntime="24.880191ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:16.210589097 +0000 UTC m=+139.936959406" watchObservedRunningTime="2026-07-16 04:41:16.21775397 +0000 UTC m=+139.944124269" Jul 16 04:41:17.186460 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.186425 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:17.300492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.300459 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:41:17.303947 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.303930 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.329399 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.329362 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:41:17.424851 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.424815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.424993 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.424859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.424993 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.424891 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd8k5\" (UniqueName: \"kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.424993 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.424934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.425107 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.424999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.425107 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.425057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.425107 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.425099 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526384 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526444 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd8k5\" (UniqueName: \"kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.526666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.526557 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.527285 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.527251 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.527421 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.527317 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.527496 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.527443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.527587 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.527569 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.528830 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.528809 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.528965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.528950 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.538282 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.538261 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd8k5\" (UniqueName: \"kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5\") pod \"console-796748bf6c-rn8t8\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.612843 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.612809 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:17.776653 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:17.776575 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:41:17.779340 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:41:17.779312 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a5b7bb_789e_43d8_bcea_d822322d6cfe.slice/crio-2e9f90ba5ecc0aec511a852d1ffe0d32131fc2b3e747256e443e4b6e12bed660 WatchSource:0}: Error finding container 2e9f90ba5ecc0aec511a852d1ffe0d32131fc2b3e747256e443e4b6e12bed660: Status 404 returned error can't find the container with id 2e9f90ba5ecc0aec511a852d1ffe0d32131fc2b3e747256e443e4b6e12bed660 Jul 16 04:41:18.127997 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:18.127963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796748bf6c-rn8t8" event={"ID":"78a5b7bb-789e-43d8-bcea-d822322d6cfe","Type":"ContainerStarted","Data":"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230"} Jul 16 04:41:18.128170 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:18.128056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796748bf6c-rn8t8" event={"ID":"78a5b7bb-789e-43d8-bcea-d822322d6cfe","Type":"ContainerStarted","Data":"2e9f90ba5ecc0aec511a852d1ffe0d32131fc2b3e747256e443e4b6e12bed660"} Jul 16 04:41:18.175950 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:18.175898 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796748bf6c-rn8t8" podStartSLOduration=1.175883858 podStartE2EDuration="1.175883858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:41:18.1745685 +0000 UTC m=+141.900938799" watchObservedRunningTime="2026-07-16 04:41:18.175883858 +0000 UTC m=+141.902254208" Jul 16 04:41:21.775837 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:21.775799 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:26.137284 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.137239 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fd685fbc5-x2z5z" podUID="a3b50f09-1334-44e3-9b67-4f9ede41d053" containerName="console" containerID="cri-o://58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586" gracePeriod=15 Jul 16 04:41:26.390774 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.390713 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:41:26.509724 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509691 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.509724 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509732 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.509936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509779 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.509936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509807 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkhq6\" (UniqueName: \"kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.509936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509841 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.509936 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.509869 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert\") pod \"a3b50f09-1334-44e3-9b67-4f9ede41d053\" (UID: \"a3b50f09-1334-44e3-9b67-4f9ede41d053\") " Jul 16 04:41:26.510296 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.510263 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config" (OuterVolumeSpecName: "console-config") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:26.510296 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.510284 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca" (OuterVolumeSpecName: "service-ca") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:26.510479 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.510399 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:26.512105 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.512081 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:26.512105 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.512095 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:26.512105 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.512099 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6" (OuterVolumeSpecName: "kube-api-access-jkhq6") pod "a3b50f09-1334-44e3-9b67-4f9ede41d053" (UID: "a3b50f09-1334-44e3-9b67-4f9ede41d053"). InnerVolumeSpecName "kube-api-access-jkhq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:26.610705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610669 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:26.610705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610699 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:26.610705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610710 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:26.610705 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610719 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:26.610970 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610728 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3b50f09-1334-44e3-9b67-4f9ede41d053-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:26.610970 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:26.610736 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkhq6\" (UniqueName: \"kubernetes.io/projected/a3b50f09-1334-44e3-9b67-4f9ede41d053-kube-api-access-jkhq6\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:27.162902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.162867 2573 generic.go:356] "Generic (PLEG): container finished" podID="a3b50f09-1334-44e3-9b67-4f9ede41d053" containerID="58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586" exitCode=0 Jul 16 04:41:27.163283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.162913 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd685fbc5-x2z5z" Jul 16 04:41:27.163283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.162929 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd685fbc5-x2z5z" event={"ID":"a3b50f09-1334-44e3-9b67-4f9ede41d053","Type":"ContainerDied","Data":"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586"} Jul 16 04:41:27.163283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.162980 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd685fbc5-x2z5z" event={"ID":"a3b50f09-1334-44e3-9b67-4f9ede41d053","Type":"ContainerDied","Data":"273c456d19ded80b82ea764ad77e72c46764f6aca720010e0e2fe9d992ced327"} Jul 16 04:41:27.163283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.163006 2573 scope.go:117] "RemoveContainer" containerID="58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586" Jul 16 04:41:27.172024 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.172004 2573 scope.go:117] "RemoveContainer" containerID="58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586" Jul 16 04:41:27.172278 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:27.172259 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586\": container with ID starting with 58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586 not found: ID does not exist" containerID="58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586" Jul 16 04:41:27.172322 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.172285 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586"} err="failed to get container status \"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586\": rpc error: code = NotFound desc = could not find container \"58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586\": container with ID starting with 58ef0b00b8d67d86c96b0bd284201cffbe8bc84192d2bd35fd8f232c5f58f586 not found: ID does not exist" Jul 16 04:41:27.192959 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.192936 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:41:27.196451 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.196431 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fd685fbc5-x2z5z"] Jul 16 04:41:27.512478 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.512405 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b50f09-1334-44e3-9b67-4f9ede41d053" path="/var/lib/kubelet/pods/a3b50f09-1334-44e3-9b67-4f9ede41d053/volumes" Jul 16 04:41:27.613528 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.613490 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:27.613740 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.613551 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:27.618347 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.618321 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:27.734845 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:27.734817 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v82sc_5d722f5c-1317-4c46-aee0-ccabafad67a2/serve-healthcheck-canary/0.log" Jul 16 04:41:28.172343 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:28.172314 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:41:28.247551 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:28.247521 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:41:29.810947 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:29.810902 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" containerID="cri-o://180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f" gracePeriod=30 Jul 16 04:41:29.925400 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:29.925367 2573 patch_prober.go:28] interesting pod/image-registry-77fd64958c-kvp52 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body= Jul 16 04:41:29.925577 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:29.925426 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" Jul 16 04:41:30.062850 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.062794 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:41:30.141979 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.141949 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142024 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgp7v\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142055 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142082 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142112 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142173 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142205 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.142454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142422 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:30.142564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142531 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:30.142564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.142545 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted\") pod \"6512206d-f4f9-4808-a142-cfea97696618\" (UID: \"6512206d-f4f9-4808-a142-cfea97696618\") " Jul 16 04:41:30.143053 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.143028 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-registry-certificates\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.143177 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.143058 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6512206d-f4f9-4808-a142-cfea97696618-trusted-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.144601 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.144567 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:30.144700 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.144625 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:30.144700 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.144655 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:30.144812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.144742 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:30.144812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.144762 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v" (OuterVolumeSpecName: "kube-api-access-jgp7v") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "kube-api-access-jgp7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:30.151022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.150999 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6512206d-f4f9-4808-a142-cfea97696618" (UID: "6512206d-f4f9-4808-a142-cfea97696618"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:41:30.177985 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.177951 2573 generic.go:356] "Generic (PLEG): container finished" podID="6512206d-f4f9-4808-a142-cfea97696618" containerID="180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f" exitCode=0 Jul 16 04:41:30.178104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.178000 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" Jul 16 04:41:30.178104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.178035 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" event={"ID":"6512206d-f4f9-4808-a142-cfea97696618","Type":"ContainerDied","Data":"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f"} Jul 16 04:41:30.178104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.178090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fd64958c-kvp52" event={"ID":"6512206d-f4f9-4808-a142-cfea97696618","Type":"ContainerDied","Data":"5e7697cbfb9935f411f35825425efaa5b94edcc05f01af29344c49a9d4b9be22"} Jul 16 04:41:30.178231 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.178112 2573 scope.go:117] "RemoveContainer" containerID="180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f" Jul 16 04:41:30.188813 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.188797 2573 scope.go:117] "RemoveContainer" containerID="180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f" Jul 16 04:41:30.189072 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:30.189053 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f\": container with ID starting with 180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f not found: ID does not exist" containerID="180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f" Jul 16 04:41:30.189113 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.189079 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f"} err="failed to get container status \"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f\": rpc error: code = NotFound desc = could not find container \"180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f\": container with ID starting with 180ca70d5cc0722c83a0127222da9fce9cd965d0923aef6bbbe1c2b8046c150f not found: ID does not exist" Jul 16 04:41:30.211156 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.211124 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:41:30.218263 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.218244 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77fd64958c-kvp52"] Jul 16 04:41:30.243973 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.243953 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgp7v\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-kube-api-access-jgp7v\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.243973 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.243971 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-bound-sa-token\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.244065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.243981 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6512206d-f4f9-4808-a142-cfea97696618-registry-tls\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.244065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.243990 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-installation-pull-secrets\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.244065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.243998 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6512206d-f4f9-4808-a142-cfea97696618-image-registry-private-configuration\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:30.244065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:30.244007 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6512206d-f4f9-4808-a142-cfea97696618-ca-trust-extracted\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:31.517078 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:31.517045 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6512206d-f4f9-4808-a142-cfea97696618" path="/var/lib/kubelet/pods/6512206d-f4f9-4808-a142-cfea97696618/volumes" Jul 16 04:41:32.203322 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:32.203286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:32.203526 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:32.203349 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:35.200560 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:35.200471 2573 generic.go:356] "Generic (PLEG): container finished" podID="f84a151f-dc0a-4e8b-80e4-37aeb8c44a83" containerID="f04d782d96743fd967224f0437ea1d2604bc7d3d4db3a376130f97608aeebb68" exitCode=0 Jul 16 04:41:35.200895 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:35.200546 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" event={"ID":"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83","Type":"ContainerDied","Data":"f04d782d96743fd967224f0437ea1d2604bc7d3d4db3a376130f97608aeebb68"} Jul 16 04:41:35.200895 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:35.200869 2573 scope.go:117] "RemoveContainer" containerID="f04d782d96743fd967224f0437ea1d2604bc7d3d4db3a376130f97608aeebb68" Jul 16 04:41:36.206299 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:36.206260 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-qsxrr" event={"ID":"f84a151f-dc0a-4e8b-80e4-37aeb8c44a83","Type":"ContainerStarted","Data":"09459bf337a990c1683f2257ac4dfcc852607d01847957d26882030d98238ded"} Jul 16 04:41:42.205989 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.205921 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84bb6f4c75-4zn26" podUID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" containerName="console" containerID="cri-o://4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e" gracePeriod=15 Jul 16 04:41:42.467113 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.467054 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:42.554055 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554021 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554075 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554107 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554134 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554199 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7psw\" (UniqueName: \"kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554434 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554277 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554434 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554305 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config\") pod \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\" (UID: \"9abb69dc-e4da-43bd-ae44-ae0e427436d5\") " Jul 16 04:41:42.554536 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554488 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:42.554694 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554661 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:42.554694 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554688 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.554821 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554706 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config" (OuterVolumeSpecName: "console-config") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:42.554821 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.554741 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:42.556447 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.556412 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw" (OuterVolumeSpecName: "kube-api-access-q7psw") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "kube-api-access-q7psw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:42.556558 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.556515 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:42.556610 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.556555 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9abb69dc-e4da-43bd-ae44-ae0e427436d5" (UID: "9abb69dc-e4da-43bd-ae44-ae0e427436d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:42.655706 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655671 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.655706 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655702 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.655706 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655711 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.655944 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655720 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9abb69dc-e4da-43bd-ae44-ae0e427436d5-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.655944 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655728 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9abb69dc-e4da-43bd-ae44-ae0e427436d5-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:42.655944 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:42.655736 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7psw\" (UniqueName: \"kubernetes.io/projected/9abb69dc-e4da-43bd-ae44-ae0e427436d5-kube-api-access-q7psw\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:43.237691 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.237652 2573 generic.go:356] "Generic (PLEG): container finished" podID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" containerID="4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e" exitCode=0 Jul 16 04:41:43.238080 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.237723 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb6f4c75-4zn26" Jul 16 04:41:43.238080 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.237742 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb6f4c75-4zn26" event={"ID":"9abb69dc-e4da-43bd-ae44-ae0e427436d5","Type":"ContainerDied","Data":"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e"} Jul 16 04:41:43.238080 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.237825 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb6f4c75-4zn26" event={"ID":"9abb69dc-e4da-43bd-ae44-ae0e427436d5","Type":"ContainerDied","Data":"d5ac049c7c1638e987a0b75526f37898b0fc36d030a8262534b3127e6c9cb083"} Jul 16 04:41:43.238080 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.237848 2573 scope.go:117] "RemoveContainer" containerID="4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e" Jul 16 04:41:43.246545 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.246529 2573 scope.go:117] "RemoveContainer" containerID="4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e" Jul 16 04:41:43.246803 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:43.246787 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e\": container with ID starting with 4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e not found: ID does not exist" containerID="4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e" Jul 16 04:41:43.246843 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.246811 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e"} err="failed to get container status \"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e\": rpc error: code = NotFound desc = could not find container \"4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e\": container with ID starting with 4b202009b21ccfc477c4155a458e157d754647f1661ac26da1d79b57724f3b4e not found: ID does not exist" Jul 16 04:41:43.265139 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.265113 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:43.268778 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.268757 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84bb6f4c75-4zn26"] Jul 16 04:41:43.512655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:43.512577 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" path="/var/lib/kubelet/pods/9abb69dc-e4da-43bd-ae44-ae0e427436d5/volumes" Jul 16 04:41:52.208216 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:52.208121 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:52.212081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:52.212056 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-688d44c87-fp6dc" Jul 16 04:41:53.270087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.270044 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cdcc4d4b6-jvd6h" podUID="03564964-f6da-4692-8c99-9aabd7c61a20" containerName="console" containerID="cri-o://d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a" gracePeriod=15 Jul 16 04:41:53.532349 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.532290 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:53.651906 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.651865 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.651906 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.651904 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652124 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.651953 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652124 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652050 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652257 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652135 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652257 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652222 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6dgm\" (UniqueName: \"kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652361 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652271 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert\") pod \"03564964-f6da-4692-8c99-9aabd7c61a20\" (UID: \"03564964-f6da-4692-8c99-9aabd7c61a20\") " Jul 16 04:41:53.652361 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652305 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:53.652492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652470 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca" (OuterVolumeSpecName: "service-ca") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:53.652549 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652490 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config" (OuterVolumeSpecName: "console-config") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:53.652549 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652536 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.652659 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652546 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:41:53.652659 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.652553 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.654395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.654367 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:53.654395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.654386 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:41:53.654527 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.654414 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm" (OuterVolumeSpecName: "kube-api-access-t6dgm") pod "03564964-f6da-4692-8c99-9aabd7c61a20" (UID: "03564964-f6da-4692-8c99-9aabd7c61a20"). InnerVolumeSpecName "kube-api-access-t6dgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:41:53.753137 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.753097 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.753137 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.753129 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.753137 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.753178 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6dgm\" (UniqueName: \"kubernetes.io/projected/03564964-f6da-4692-8c99-9aabd7c61a20-kube-api-access-t6dgm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.753403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.753191 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/03564964-f6da-4692-8c99-9aabd7c61a20-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:53.753403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:53.753203 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/03564964-f6da-4692-8c99-9aabd7c61a20-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:41:54.285005 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.284973 2573 generic.go:356] "Generic (PLEG): container finished" podID="03564964-f6da-4692-8c99-9aabd7c61a20" containerID="d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a" exitCode=0 Jul 16 04:41:54.285429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.285013 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cdcc4d4b6-jvd6h" Jul 16 04:41:54.285429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.285024 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdcc4d4b6-jvd6h" event={"ID":"03564964-f6da-4692-8c99-9aabd7c61a20","Type":"ContainerDied","Data":"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a"} Jul 16 04:41:54.285429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.285064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cdcc4d4b6-jvd6h" event={"ID":"03564964-f6da-4692-8c99-9aabd7c61a20","Type":"ContainerDied","Data":"1a956ce3e3398e67ce20bbba5e73176188f6b85c8de956cd6c9b77f73f596547"} Jul 16 04:41:54.285429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.285086 2573 scope.go:117] "RemoveContainer" containerID="d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a" Jul 16 04:41:54.293569 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.293549 2573 scope.go:117] "RemoveContainer" containerID="d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a" Jul 16 04:41:54.293825 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:41:54.293804 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a\": container with ID starting with d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a not found: ID does not exist" containerID="d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a" Jul 16 04:41:54.293891 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.293833 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a"} err="failed to get container status \"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a\": rpc error: code = NotFound desc = could not find container \"d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a\": container with ID starting with d8699a23232eb4bfbdbc60f7913a415adf8b0cd80784bf9b8230631ecbce152a not found: ID does not exist" Jul 16 04:41:54.314172 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.314124 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:41:54.320553 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:54.320529 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cdcc4d4b6-jvd6h"] Jul 16 04:41:55.512894 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:41:55.512859 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03564964-f6da-4692-8c99-9aabd7c61a20" path="/var/lib/kubelet/pods/03564964-f6da-4692-8c99-9aabd7c61a20/volumes" Jul 16 04:42:28.123737 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.123699 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124058 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="alertmanager" containerID="cri-o://47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" gracePeriod=120 Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124124 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="prom-label-proxy" containerID="cri-o://90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" gracePeriod=120 Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124129 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-metric" containerID="cri-o://944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" gracePeriod=120 Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124202 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="config-reloader" containerID="cri-o://80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" gracePeriod=120 Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124185 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-web" containerID="cri-o://17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" gracePeriod=120 Jul 16 04:42:28.124362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.124124 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy" containerID="cri-o://e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" gracePeriod=120 Jul 16 04:42:28.415342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415254 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" exitCode=0 Jul 16 04:42:28.415342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415278 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" exitCode=0 Jul 16 04:42:28.415342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415286 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" exitCode=0 Jul 16 04:42:28.415342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415292 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" exitCode=0 Jul 16 04:42:28.415342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415329 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390"} Jul 16 04:42:28.415621 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415393 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b"} Jul 16 04:42:28.415621 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f"} Jul 16 04:42:28.415621 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:28.415443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca"} Jul 16 04:42:29.393269 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.393239 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.422663 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422622 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" exitCode=0 Jul 16 04:42:29.422663 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422657 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerID="17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" exitCode=0 Jul 16 04:42:29.422861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422706 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9"} Jul 16 04:42:29.422861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6"} Jul 16 04:42:29.422861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422719 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.422861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422821 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c4c318e2-7d54-4acf-ba31-00af99b59d87","Type":"ContainerDied","Data":"1f94e6e5a752f38855289c5658e38ff65689c4ec0d8b5b3e4cef789bd8150ac0"} Jul 16 04:42:29.422861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.422853 2573 scope.go:117] "RemoveContainer" containerID="90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" Jul 16 04:42:29.431213 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.431193 2573 scope.go:117] "RemoveContainer" containerID="944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" Jul 16 04:42:29.440777 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.440757 2573 scope.go:117] "RemoveContainer" containerID="e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" Jul 16 04:42:29.450128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.450106 2573 scope.go:117] "RemoveContainer" containerID="17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" Jul 16 04:42:29.457882 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.457858 2573 scope.go:117] "RemoveContainer" containerID="80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" Jul 16 04:42:29.465529 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.465505 2573 scope.go:117] "RemoveContainer" containerID="47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" Jul 16 04:42:29.474453 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.474430 2573 scope.go:117] "RemoveContainer" containerID="888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8" Jul 16 04:42:29.482213 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.482187 2573 scope.go:117] "RemoveContainer" containerID="90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" Jul 16 04:42:29.482522 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.482499 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390\": container with ID starting with 90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390 not found: ID does not exist" containerID="90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" Jul 16 04:42:29.482625 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.482534 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390"} err="failed to get container status \"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390\": rpc error: code = NotFound desc = could not find container \"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390\": container with ID starting with 90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390 not found: ID does not exist" Jul 16 04:42:29.482625 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.482560 2573 scope.go:117] "RemoveContainer" containerID="944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" Jul 16 04:42:29.482866 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.482848 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9\": container with ID starting with 944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9 not found: ID does not exist" containerID="944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" Jul 16 04:42:29.482902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.482874 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9"} err="failed to get container status \"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9\": rpc error: code = NotFound desc = could not find container \"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9\": container with ID starting with 944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9 not found: ID does not exist" Jul 16 04:42:29.482902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.482891 2573 scope.go:117] "RemoveContainer" containerID="e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" Jul 16 04:42:29.483132 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.483116 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b\": container with ID starting with e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b not found: ID does not exist" containerID="e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" Jul 16 04:42:29.483248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483137 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b"} err="failed to get container status \"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b\": rpc error: code = NotFound desc = could not find container \"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b\": container with ID starting with e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b not found: ID does not exist" Jul 16 04:42:29.483248 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483244 2573 scope.go:117] "RemoveContainer" containerID="17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" Jul 16 04:42:29.483481 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.483458 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6\": container with ID starting with 17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6 not found: ID does not exist" containerID="17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" Jul 16 04:42:29.483543 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483490 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6"} err="failed to get container status \"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6\": rpc error: code = NotFound desc = could not find container \"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6\": container with ID starting with 17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6 not found: ID does not exist" Jul 16 04:42:29.483543 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483512 2573 scope.go:117] "RemoveContainer" containerID="80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" Jul 16 04:42:29.483763 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.483746 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f\": container with ID starting with 80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f not found: ID does not exist" containerID="80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" Jul 16 04:42:29.483810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483768 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f"} err="failed to get container status \"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f\": rpc error: code = NotFound desc = could not find container \"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f\": container with ID starting with 80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f not found: ID does not exist" Jul 16 04:42:29.483810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.483787 2573 scope.go:117] "RemoveContainer" containerID="47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" Jul 16 04:42:29.484048 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.484026 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca\": container with ID starting with 47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca not found: ID does not exist" containerID="47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" Jul 16 04:42:29.484129 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484056 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca"} err="failed to get container status \"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca\": rpc error: code = NotFound desc = could not find container \"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca\": container with ID starting with 47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca not found: ID does not exist" Jul 16 04:42:29.484129 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484080 2573 scope.go:117] "RemoveContainer" containerID="888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8" Jul 16 04:42:29.484331 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:42:29.484313 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8\": container with ID starting with 888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8 not found: ID does not exist" containerID="888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8" Jul 16 04:42:29.484370 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484340 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8"} err="failed to get container status \"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8\": rpc error: code = NotFound desc = could not find container \"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8\": container with ID starting with 888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8 not found: ID does not exist" Jul 16 04:42:29.484370 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484362 2573 scope.go:117] "RemoveContainer" containerID="90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390" Jul 16 04:42:29.484568 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484548 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390"} err="failed to get container status \"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390\": rpc error: code = NotFound desc = could not find container \"90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390\": container with ID starting with 90b2bf7ad1089e973b783f6e6b2467d118a94dc83a1d893c162bde2ba8623390 not found: ID does not exist" Jul 16 04:42:29.484607 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484569 2573 scope.go:117] "RemoveContainer" containerID="944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9" Jul 16 04:42:29.484794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484764 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9"} err="failed to get container status \"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9\": rpc error: code = NotFound desc = could not find container \"944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9\": container with ID starting with 944daf31e2e561d6c01f693ec2804a033bbdc2d0ccac22f7a462aec3052b2bf9 not found: ID does not exist" Jul 16 04:42:29.484863 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.484797 2573 scope.go:117] "RemoveContainer" containerID="e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b" Jul 16 04:42:29.485041 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485020 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b"} err="failed to get container status \"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b\": rpc error: code = NotFound desc = could not find container \"e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b\": container with ID starting with e602c7b8f585de10dcb557080708189cb3b8d12ca5fafe324808c14a6f1f582b not found: ID does not exist" Jul 16 04:42:29.485088 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485042 2573 scope.go:117] "RemoveContainer" containerID="17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6" Jul 16 04:42:29.485275 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485254 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6"} err="failed to get container status \"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6\": rpc error: code = NotFound desc = could not find container \"17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6\": container with ID starting with 17c07322da9d1fedbb54ba34daa3fffb8c51bcfd6719fc240531f51a1dab58a6 not found: ID does not exist" Jul 16 04:42:29.485325 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485277 2573 scope.go:117] "RemoveContainer" containerID="80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f" Jul 16 04:42:29.485463 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485437 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f"} err="failed to get container status \"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f\": rpc error: code = NotFound desc = could not find container \"80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f\": container with ID starting with 80b8d7db9b61a2427e750ecac8e35daf5bf75243496e8f9c4dc3d6c545d5040f not found: ID does not exist" Jul 16 04:42:29.485509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485464 2573 scope.go:117] "RemoveContainer" containerID="47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca" Jul 16 04:42:29.485661 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485641 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca"} err="failed to get container status \"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca\": rpc error: code = NotFound desc = could not find container \"47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca\": container with ID starting with 47bb08260c7e9616459e2215b17f6889c8f56680ec44bd6534cc4e2559b59eca not found: ID does not exist" Jul 16 04:42:29.485718 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485661 2573 scope.go:117] "RemoveContainer" containerID="888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8" Jul 16 04:42:29.485861 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.485842 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8"} err="failed to get container status \"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8\": rpc error: code = NotFound desc = could not find container \"888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8\": container with ID starting with 888fef1f339483d80190431c4f477f88613f9a4b951a18c05718d76153569ee8 not found: ID does not exist" Jul 16 04:42:29.554993 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.554963 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555185 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555017 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555185 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555040 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555185 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555070 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555374 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555207 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555374 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555240 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555374 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555267 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555530 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555431 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555530 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555519 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftxmh\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555554 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555585 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555616 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.555781 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555667 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca\") pod \"c4c318e2-7d54-4acf-ba31-00af99b59d87\" (UID: \"c4c318e2-7d54-4acf-ba31-00af99b59d87\") " Jul 16 04:42:29.556442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.555580 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:42:29.556442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.556060 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:42:29.556442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.556262 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:42:29.558075 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558042 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume" (OuterVolumeSpecName: "config-volume") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.558340 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558314 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.558431 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558405 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out" (OuterVolumeSpecName: "config-out") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:42:29.558665 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558638 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh" (OuterVolumeSpecName: "kube-api-access-ftxmh") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "kube-api-access-ftxmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:42:29.558759 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558730 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.558820 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.558749 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.559663 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.559637 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:42:29.559776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.559758 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.562440 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.562354 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.569329 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.569300 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config" (OuterVolumeSpecName: "web-config") pod "c4c318e2-7d54-4acf-ba31-00af99b59d87" (UID: "c4c318e2-7d54-4acf-ba31-00af99b59d87"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:42:29.656831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656778 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-main-tls\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.656831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656822 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.656831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656834 2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-cluster-tls-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.656831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656842 2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-volume\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656851 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656872 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-main-db\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656883 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656891 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftxmh\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-kube-api-access-ftxmh\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656899 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4c318e2-7d54-4acf-ba31-00af99b59d87-tls-assets\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656909 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656916 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4c318e2-7d54-4acf-ba31-00af99b59d87-web-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656924 2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4c318e2-7d54-4acf-ba31-00af99b59d87-metrics-client-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.657095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.656932 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4c318e2-7d54-4acf-ba31-00af99b59d87-config-out\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:42:29.752137 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.752104 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:29.756517 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.756491 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:29.798923 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.798893 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:29.799191 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799177 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-metric" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799194 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-metric" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799203 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="prom-label-proxy" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799208 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="prom-label-proxy" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799220 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="init-config-reloader" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799226 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="init-config-reloader" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799232 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="alertmanager" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799237 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="alertmanager" Jul 16 04:42:29.799242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799243 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="config-reloader" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799248 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="config-reloader" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799260 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b50f09-1334-44e3-9b67-4f9ede41d053" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799266 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b50f09-1334-44e3-9b67-4f9ede41d053" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799275 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03564964-f6da-4692-8c99-9aabd7c61a20" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799280 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="03564964-f6da-4692-8c99-9aabd7c61a20" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799286 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799291 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799297 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799302 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799309 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-web" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799314 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-web" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799320 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799324 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799367 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799373 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6512206d-f4f9-4808-a142-cfea97696618" containerName="registry" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799379 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="prom-label-proxy" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799386 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-web" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799393 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="kube-rbac-proxy-metric" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799397 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="alertmanager" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799403 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b50f09-1334-44e3-9b67-4f9ede41d053" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799409 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" containerName="config-reloader" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799414 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abb69dc-e4da-43bd-ae44-ae0e427436d5" containerName="console" Jul 16 04:42:29.799499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.799419 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="03564964-f6da-4692-8c99-9aabd7c61a20" containerName="console" Jul 16 04:42:29.804487 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.804466 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.808707 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.808686 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 04:42:29.808901 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.808888 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tcwqz\"" Jul 16 04:42:29.808955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.808896 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 04:42:29.809139 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809124 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 04:42:29.809213 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809167 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 04:42:29.809676 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809659 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 04:42:29.809756 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809659 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 04:42:29.809806 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809681 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 04:42:29.809968 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.809949 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 04:42:29.818458 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.818439 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 04:42:29.825808 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.825785 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:29.864203 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.863966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-config-out\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864267 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-config-volume\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864295 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864373 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864317 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx5bx\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-kube-api-access-kx5bx\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864501 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864391 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864501 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864501 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864480 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-web-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864592 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864592 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864544 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864592 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.864681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.864641 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965283 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-web-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965345 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965536 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965593 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-config-out\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965651 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-config-volume\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965691 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx5bx\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-kube-api-access-kx5bx\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.965812 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.965790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.966844 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.966277 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.966844 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.966458 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.966844 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.966788 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.968516 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.968475 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.968516 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.968483 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.968664 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.968631 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.968708 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.968671 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-web-config\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.969044 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.969012 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.969258 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.969232 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/006823b2-d704-4b73-a596-4b537e43de6a-config-out\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.969344 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.969290 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-config-volume\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.969541 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.969523 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.970004 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.969988 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/006823b2-d704-4b73-a596-4b537e43de6a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:29.977036 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:29.977015 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx5bx\" (UniqueName: \"kubernetes.io/projected/006823b2-d704-4b73-a596-4b537e43de6a-kube-api-access-kx5bx\") pod \"alertmanager-main-0\" (UID: \"006823b2-d704-4b73-a596-4b537e43de6a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:30.114071 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:30.114042 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 04:42:30.275063 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:30.274742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 04:42:30.277579 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:42:30.277554 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006823b2_d704_4b73_a596_4b537e43de6a.slice/crio-f9df14c98794547f0ae170a0288af618dd41d0ab7fc3f06225443b86ea365cfe WatchSource:0}: Error finding container f9df14c98794547f0ae170a0288af618dd41d0ab7fc3f06225443b86ea365cfe: Status 404 returned error can't find the container with id f9df14c98794547f0ae170a0288af618dd41d0ab7fc3f06225443b86ea365cfe Jul 16 04:42:30.428057 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:30.427965 2573 generic.go:356] "Generic (PLEG): container finished" podID="006823b2-d704-4b73-a596-4b537e43de6a" containerID="5163b969c298ddfbc980f2abf4291e36260888a501222624b2b74882ec4945c5" exitCode=0 Jul 16 04:42:30.428492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:30.428054 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerDied","Data":"5163b969c298ddfbc980f2abf4291e36260888a501222624b2b74882ec4945c5"} Jul 16 04:42:30.428492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:30.428129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"f9df14c98794547f0ae170a0288af618dd41d0ab7fc3f06225443b86ea365cfe"} Jul 16 04:42:31.435983 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.435949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"0920b9f7f251ee57ad7dd07ed0f4441197b60c8557ac685399a01e236d16e7ce"} Jul 16 04:42:31.436445 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.436010 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"5d0238a26f5df3e29c5e0cb6dcc90730e69b9f9b3c10b44d602d2e38a2baeae8"} Jul 16 04:42:31.436445 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.436036 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"b25baa1a75ec526d6f70e18e06527054750f7c23c16b9d697f3ae6c080af1e16"} Jul 16 04:42:31.436445 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.436060 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"36d2667aa4e2f73be2ffea59dc41e96c302212e1c6e80b9c0ce34e772df0ce44"} Jul 16 04:42:31.436445 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.436083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"d6ea82557b88d5ca75a9cefda459d3873aaab5da0df3185bef7a66f2063f05d5"} Jul 16 04:42:31.436445 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.436105 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"006823b2-d704-4b73-a596-4b537e43de6a","Type":"ContainerStarted","Data":"a503a620c5ebd7710a7d744afef702a709fa8985112a02d693f85340e2220198"} Jul 16 04:42:31.466653 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.466597 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.450270491 podStartE2EDuration="2.466581339s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.310848ms" isStatefulPod=false podCreationTimestamp="2026-07-16 04:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:42:31.465835269 +0000 UTC m=+215.192205567" watchObservedRunningTime="2026-07-16 04:42:31.466581339 +0000 UTC m=+215.192951638" Jul 16 04:42:31.512515 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:31.512438 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4c318e2-7d54-4acf-ba31-00af99b59d87" path="/var/lib/kubelet/pods/c4c318e2-7d54-4acf-ba31-00af99b59d87/volumes" Jul 16 04:42:42.102589 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.102549 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:42:42.107279 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.107257 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.126157 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.126121 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:42:42.180220 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180185 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180220 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180222 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szv9n\" (UniqueName: \"kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.180418 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.180387 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.281775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281741 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.281775 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281812 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szv9n\" (UniqueName: \"kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.281968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282634 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.282604 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282752 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.282645 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282752 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.282645 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.282963 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.282939 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.284313 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.284288 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.284577 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.284547 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.295038 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.295015 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szv9n\" (UniqueName: \"kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n\") pod \"console-7649768847-78gtc\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.415847 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.415807 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:42.553650 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:42.553623 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:42:42.556877 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:42:42.556848 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d9713d_8a61_412e_8244_8d193c3f5bf6.slice/crio-b251dc3d26a6891cdaf0fd391f9213cf4283139817e3679f31f0a9fbb3f41826 WatchSource:0}: Error finding container b251dc3d26a6891cdaf0fd391f9213cf4283139817e3679f31f0a9fbb3f41826: Status 404 returned error can't find the container with id b251dc3d26a6891cdaf0fd391f9213cf4283139817e3679f31f0a9fbb3f41826 Jul 16 04:42:43.487334 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:43.487293 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7649768847-78gtc" event={"ID":"d0d9713d-8a61-412e-8244-8d193c3f5bf6","Type":"ContainerStarted","Data":"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d"} Jul 16 04:42:43.487722 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:43.487351 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7649768847-78gtc" event={"ID":"d0d9713d-8a61-412e-8244-8d193c3f5bf6","Type":"ContainerStarted","Data":"b251dc3d26a6891cdaf0fd391f9213cf4283139817e3679f31f0a9fbb3f41826"} Jul 16 04:42:43.513406 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:43.513360 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7649768847-78gtc" podStartSLOduration=1.513344443 podStartE2EDuration="1.513344443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:42:43.511801721 +0000 UTC m=+227.238172024" watchObservedRunningTime="2026-07-16 04:42:43.513344443 +0000 UTC m=+227.239714741" Jul 16 04:42:52.416016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:52.415982 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:52.416414 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:52.416042 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:52.421628 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:52.421596 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:52.525918 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:52.525882 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:42:52.602931 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:42:52.602898 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:43:17.622926 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.622868 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-796748bf6c-rn8t8" podUID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" containerName="console" containerID="cri-o://15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230" gracePeriod=15 Jul 16 04:43:17.884181 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.884111 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:43:17.973941 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.973891 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.973941 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.973943 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.973962 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.973992 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974037 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd8k5\" (UniqueName: \"kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974057 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974088 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca\") pod \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\" (UID: \"78a5b7bb-789e-43d8-bcea-d822322d6cfe\") " Jul 16 04:43:17.974533 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974504 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config" (OuterVolumeSpecName: "console-config") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:43:17.974533 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974519 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:43:17.974628 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974513 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:43:17.974628 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.974552 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca" (OuterVolumeSpecName: "service-ca") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:43:17.976389 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.976348 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:43:17.976513 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.976388 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:43:17.976513 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:17.976456 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5" (OuterVolumeSpecName: "kube-api-access-zd8k5") pod "78a5b7bb-789e-43d8-bcea-d822322d6cfe" (UID: "78a5b7bb-789e-43d8-bcea-d822322d6cfe"). InnerVolumeSpecName "kube-api-access-zd8k5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:43:18.075246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075189 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd8k5\" (UniqueName: \"kubernetes.io/projected/78a5b7bb-789e-43d8-bcea-d822322d6cfe-kube-api-access-zd8k5\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075238 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075249 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075261 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075269 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075277 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78a5b7bb-789e-43d8-bcea-d822322d6cfe-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.075554 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.075285 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78a5b7bb-789e-43d8-bcea-d822322d6cfe-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:18.622835 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.622802 2573 generic.go:356] "Generic (PLEG): container finished" podID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" containerID="15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230" exitCode=0 Jul 16 04:43:18.623016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.622847 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796748bf6c-rn8t8" Jul 16 04:43:18.623016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.622878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796748bf6c-rn8t8" event={"ID":"78a5b7bb-789e-43d8-bcea-d822322d6cfe","Type":"ContainerDied","Data":"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230"} Jul 16 04:43:18.623016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.622919 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796748bf6c-rn8t8" event={"ID":"78a5b7bb-789e-43d8-bcea-d822322d6cfe","Type":"ContainerDied","Data":"2e9f90ba5ecc0aec511a852d1ffe0d32131fc2b3e747256e443e4b6e12bed660"} Jul 16 04:43:18.623016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.622940 2573 scope.go:117] "RemoveContainer" containerID="15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230" Jul 16 04:43:18.631550 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.631531 2573 scope.go:117] "RemoveContainer" containerID="15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230" Jul 16 04:43:18.631810 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:43:18.631790 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230\": container with ID starting with 15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230 not found: ID does not exist" containerID="15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230" Jul 16 04:43:18.631862 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.631819 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230"} err="failed to get container status \"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230\": rpc error: code = NotFound desc = could not find container \"15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230\": container with ID starting with 15779b333b800948a402013e3d5ef92ae24bfd54b80c788a353c71b2287c5230 not found: ID does not exist" Jul 16 04:43:18.644467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.644440 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:43:18.650514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:18.650489 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-796748bf6c-rn8t8"] Jul 16 04:43:19.512879 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:19.512806 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" path="/var/lib/kubelet/pods/78a5b7bb-789e-43d8-bcea-d822322d6cfe/volumes" Jul 16 04:43:24.375713 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.375680 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx"] Jul 16 04:43:24.376065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.375994 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" containerName="console" Jul 16 04:43:24.376065 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.376005 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" containerName="console" Jul 16 04:43:24.376170 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.376070 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="78a5b7bb-789e-43d8-bcea-d822322d6cfe" containerName="console" Jul 16 04:43:24.380327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.380308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.383927 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.383901 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 04:43:24.384064 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.384042 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w5jxj\"" Jul 16 04:43:24.384827 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.384810 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 04:43:24.394669 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.394645 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx"] Jul 16 04:43:24.427623 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.427600 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.427740 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.427654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.427740 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.427675 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b75dn\" (UniqueName: \"kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.528262 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.528234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.528388 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.528268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b75dn\" (UniqueName: \"kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.528388 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.528321 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.528605 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.528586 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.528646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.528612 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.540442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.540415 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b75dn\" (UniqueName: \"kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.690061 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.689979 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:24.820840 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:24.820798 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx"] Jul 16 04:43:24.823007 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:43:24.822977 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce21ee1_1cf4_4481_aae3_bacdf3d3d9cc.slice/crio-68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8 WatchSource:0}: Error finding container 68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8: Status 404 returned error can't find the container with id 68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8 Jul 16 04:43:25.653610 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:25.653568 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" event={"ID":"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc","Type":"ContainerStarted","Data":"68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8"} Jul 16 04:43:32.685328 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:32.685296 2573 generic.go:356] "Generic (PLEG): container finished" podID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerID="13f36c9beb4013a9d95ed901ad5339dbb4a91c301066012f584d0d5761655e44" exitCode=0 Jul 16 04:43:32.685720 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:32.685362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" event={"ID":"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc","Type":"ContainerDied","Data":"13f36c9beb4013a9d95ed901ad5339dbb4a91c301066012f584d0d5761655e44"} Jul 16 04:43:35.699186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:35.699122 2573 generic.go:356] "Generic (PLEG): container finished" podID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerID="aa5b3f749ee142c1cf5b1fd69a5f51446db6dc384236e8a570140691c4c1ac59" exitCode=0 Jul 16 04:43:35.699186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:35.699179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" event={"ID":"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc","Type":"ContainerDied","Data":"aa5b3f749ee142c1cf5b1fd69a5f51446db6dc384236e8a570140691c4c1ac59"} Jul 16 04:43:43.736621 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:43.736583 2573 generic.go:356] "Generic (PLEG): container finished" podID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerID="86b4c0f19dd6bc4cd84d4bc791be01569b2b56c5a84ea821b702383850da0533" exitCode=0 Jul 16 04:43:43.736621 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:43.736623 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" event={"ID":"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc","Type":"ContainerDied","Data":"86b4c0f19dd6bc4cd84d4bc791be01569b2b56c5a84ea821b702383850da0533"} Jul 16 04:43:44.895230 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:44.895196 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:45.012831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.012749 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b75dn\" (UniqueName: \"kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn\") pod \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " Jul 16 04:43:45.012831 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.012812 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util\") pod \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " Jul 16 04:43:45.013021 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.012908 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle\") pod \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\" (UID: \"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc\") " Jul 16 04:43:45.013550 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.013516 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle" (OuterVolumeSpecName: "bundle") pod "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" (UID: "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:43:45.015079 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.015043 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn" (OuterVolumeSpecName: "kube-api-access-b75dn") pod "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" (UID: "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc"). InnerVolumeSpecName "kube-api-access-b75dn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:43:45.017052 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.017033 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util" (OuterVolumeSpecName: "util") pod "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" (UID: "1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:43:45.113828 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.113783 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:45.113828 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.113822 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:45.113828 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.113836 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b75dn\" (UniqueName: \"kubernetes.io/projected/1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc-kube-api-access-b75dn\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:43:45.746352 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.746323 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" Jul 16 04:43:45.746509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.746322 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wjgjx" event={"ID":"1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc","Type":"ContainerDied","Data":"68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8"} Jul 16 04:43:45.746509 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:45.746457 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68527a726a9f9e74d4e2e194193259aeeacd959b1fd06199dbba6b2fb6fb82a8" Jul 16 04:43:53.240255 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240217 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk"] Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240556 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="extract" Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240568 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="extract" Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240579 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="util" Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240584 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="util" Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240597 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="pull" Jul 16 04:43:53.240644 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240606 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="pull" Jul 16 04:43:53.240826 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.240664 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ce21ee1-1cf4-4481-aae3-bacdf3d3d9cc" containerName="extract" Jul 16 04:43:53.244086 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.244068 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.250955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.250931 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:43:53.251089 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.250960 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 04:43:53.251089 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.251000 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-swrq5\"" Jul 16 04:43:53.268183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.268136 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk"] Jul 16 04:43:53.383846 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.383806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbec1d3e-4fef-433c-87eb-5f1c318261b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.384038 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.383860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6cwv\" (UniqueName: \"kubernetes.io/projected/cbec1d3e-4fef-433c-87eb-5f1c318261b2-kube-api-access-t6cwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.485259 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.485214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6cwv\" (UniqueName: \"kubernetes.io/projected/cbec1d3e-4fef-433c-87eb-5f1c318261b2-kube-api-access-t6cwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.485457 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.485362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbec1d3e-4fef-433c-87eb-5f1c318261b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.489613 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.485820 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbec1d3e-4fef-433c-87eb-5f1c318261b2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.496717 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.496655 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6cwv\" (UniqueName: \"kubernetes.io/projected/cbec1d3e-4fef-433c-87eb-5f1c318261b2-kube-api-access-t6cwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-b52xk\" (UID: \"cbec1d3e-4fef-433c-87eb-5f1c318261b2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.553750 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.553705 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" Jul 16 04:43:53.690377 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.690314 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk"] Jul 16 04:43:53.693059 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:43:53.693028 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbec1d3e_4fef_433c_87eb_5f1c318261b2.slice/crio-8a32b9e150ee9249556702d74cabd6bce3408fcdd8f620419dda9e5dff045e68 WatchSource:0}: Error finding container 8a32b9e150ee9249556702d74cabd6bce3408fcdd8f620419dda9e5dff045e68: Status 404 returned error can't find the container with id 8a32b9e150ee9249556702d74cabd6bce3408fcdd8f620419dda9e5dff045e68 Jul 16 04:43:53.778419 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:53.778339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" event={"ID":"cbec1d3e-4fef-433c-87eb-5f1c318261b2","Type":"ContainerStarted","Data":"8a32b9e150ee9249556702d74cabd6bce3408fcdd8f620419dda9e5dff045e68"} Jul 16 04:43:55.789662 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:55.789618 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" event={"ID":"cbec1d3e-4fef-433c-87eb-5f1c318261b2","Type":"ContainerStarted","Data":"c23eb4b04d9e35936dabd135f454121b319ab1dde2be87d2b0d47df633ab36b3"} Jul 16 04:43:56.563940 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.563887 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b52xk" podStartSLOduration=2.118483072 podStartE2EDuration="3.563871784s" totalImagesPullingTime="1.445388712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:43:55.821041563 +0000 UTC m=+299.547411863" watchObservedRunningTime="2026-07-16 04:43:56.563871784 +0000 UTC m=+300.290242082" Jul 16 04:43:56.565822 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.565793 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf"] Jul 16 04:43:56.569350 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.569329 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.571622 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.571603 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 04:43:56.572022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.572003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w5jxj\"" Jul 16 04:43:56.572635 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.572616 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 04:43:56.609317 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.609289 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf"] Jul 16 04:43:56.613982 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.613949 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.614136 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.613993 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frgf\" (UniqueName: \"kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.614136 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.614048 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.714523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.714490 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.714523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.714528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frgf\" (UniqueName: \"kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.714791 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.714559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.714981 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.714930 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.715082 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.714984 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.719863 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.719843 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:43:56.723128 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.723110 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:43:56.727744 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.727725 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 04:43:56.735194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.735167 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 04:43:56.746360 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.746337 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 04:43:56.755899 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.755868 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frgf\" (UniqueName: \"kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:56.883366 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.883334 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w5jxj\"" Jul 16 04:43:56.888461 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:56.888438 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:43:57.030084 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.030058 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf"] Jul 16 04:43:57.032072 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:43:57.032037 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb38b58_b6d4_43dc_8fb7_5bfe1e7f7912.slice/crio-ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900 WatchSource:0}: Error finding container ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900: Status 404 returned error can't find the container with id ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900 Jul 16 04:43:57.125918 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.125882 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g2f4h"] Jul 16 04:43:57.129246 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.129223 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.132349 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.132328 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6qdkw\"" Jul 16 04:43:57.132630 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.132604 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 04:43:57.132739 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.132720 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 04:43:57.158515 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.158432 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g2f4h"] Jul 16 04:43:57.218348 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.218315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.218487 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.218408 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpws8\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-kube-api-access-fpws8\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.319488 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.319449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpws8\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-kube-api-access-fpws8\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.319654 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.319509 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.328446 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.328415 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpws8\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-kube-api-access-fpws8\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.328630 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.328612 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f9acdc15-2db6-4f82-8ca4-d8bce4efc676-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-g2f4h\" (UID: \"f9acdc15-2db6-4f82-8ca4-d8bce4efc676\") " pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.499422 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.499328 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:43:57.633088 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.633031 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-g2f4h"] Jul 16 04:43:57.635673 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:43:57.635652 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9acdc15_2db6_4f82_8ca4_d8bce4efc676.slice/crio-a5feb98d5d91187b845504c806e7174d7cd3246b6ea434854d3877c7366a6e08 WatchSource:0}: Error finding container a5feb98d5d91187b845504c806e7174d7cd3246b6ea434854d3877c7366a6e08: Status 404 returned error can't find the container with id a5feb98d5d91187b845504c806e7174d7cd3246b6ea434854d3877c7366a6e08 Jul 16 04:43:57.798928 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.798882 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" event={"ID":"f9acdc15-2db6-4f82-8ca4-d8bce4efc676","Type":"ContainerStarted","Data":"a5feb98d5d91187b845504c806e7174d7cd3246b6ea434854d3877c7366a6e08"} Jul 16 04:43:57.800269 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.800245 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerID="b8b135b986625ee50cb88c50b32e60fa4bf14802ff64fb7253255252bcb866c5" exitCode=0 Jul 16 04:43:57.800404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.800333 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" event={"ID":"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912","Type":"ContainerDied","Data":"b8b135b986625ee50cb88c50b32e60fa4bf14802ff64fb7253255252bcb866c5"} Jul 16 04:43:57.800404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:57.800377 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" event={"ID":"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912","Type":"ContainerStarted","Data":"ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900"} Jul 16 04:43:59.639520 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.638924 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl"] Jul 16 04:43:59.644427 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.644399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.647542 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.647510 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-njphv\"" Jul 16 04:43:59.654779 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.654757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl"] Jul 16 04:43:59.740771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.740722 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz5w5\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-kube-api-access-vz5w5\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.740968 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.740861 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.841733 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.841703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz5w5\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-kube-api-access-vz5w5\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.841921 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.841768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.852680 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.852647 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.852837 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.852771 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz5w5\" (UniqueName: \"kubernetes.io/projected/910f0fa6-74fc-4810-a780-28ba78781f3d-kube-api-access-vz5w5\") pod \"cert-manager-cainjector-559cdd8c87-2nzxl\" (UID: \"910f0fa6-74fc-4810-a780-28ba78781f3d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:43:59.956569 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:43:59.956251 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" Jul 16 04:44:01.117562 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.117521 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl"] Jul 16 04:44:01.120160 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:01.120122 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod910f0fa6_74fc_4810_a780_28ba78781f3d.slice/crio-17e9b10982d333335bec13619afef40934bce896ffd00a9302fee8c49f66f0e9 WatchSource:0}: Error finding container 17e9b10982d333335bec13619afef40934bce896ffd00a9302fee8c49f66f0e9: Status 404 returned error can't find the container with id 17e9b10982d333335bec13619afef40934bce896ffd00a9302fee8c49f66f0e9 Jul 16 04:44:01.825477 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.825438 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" event={"ID":"910f0fa6-74fc-4810-a780-28ba78781f3d","Type":"ContainerStarted","Data":"dcbb97a14d62aaf96969d2595054a480fbf0128f5864f871447150e31418748e"} Jul 16 04:44:01.825477 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.825483 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" event={"ID":"910f0fa6-74fc-4810-a780-28ba78781f3d","Type":"ContainerStarted","Data":"17e9b10982d333335bec13619afef40934bce896ffd00a9302fee8c49f66f0e9"} Jul 16 04:44:01.827044 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.827013 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" event={"ID":"f9acdc15-2db6-4f82-8ca4-d8bce4efc676","Type":"ContainerStarted","Data":"46c5f8b52e1f1464996d1b34d9115f9c1312258ebf23d5a0833900db057ae2c8"} Jul 16 04:44:01.827206 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.827085 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:44:01.828896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.828869 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerID="b5433519097dacad84bb3d21cdafd3c60634b9c23238ced0bf6dc1f8cc1cfd06" exitCode=0 Jul 16 04:44:01.829011 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.828934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" event={"ID":"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912","Type":"ContainerDied","Data":"b5433519097dacad84bb3d21cdafd3c60634b9c23238ced0bf6dc1f8cc1cfd06"} Jul 16 04:44:01.844459 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.844414 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2nzxl" podStartSLOduration=2.844392884 podStartE2EDuration="2.844392884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:01.842578468 +0000 UTC m=+305.568948768" watchObservedRunningTime="2026-07-16 04:44:01.844392884 +0000 UTC m=+305.570763182" Jul 16 04:44:01.883203 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:01.883140 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" podStartSLOduration=1.550945094 podStartE2EDuration="4.88311678s" totalImagesPullingTime="3.332171686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:43:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:01.881523728 +0000 UTC m=+305.607894028" watchObservedRunningTime="2026-07-16 04:44:01.88311678 +0000 UTC m=+305.609487081" Jul 16 04:44:02.840991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:02.840949 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerID="6b95157f261ad3dac2cc0fcda26e4c3332f66165ded088d79e911fc2391a56cc" exitCode=0 Jul 16 04:44:02.841417 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:02.841032 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" event={"ID":"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912","Type":"ContainerDied","Data":"6b95157f261ad3dac2cc0fcda26e4c3332f66165ded088d79e911fc2391a56cc"} Jul 16 04:44:03.979499 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:03.979474 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:44:04.081917 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.081876 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle\") pod \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " Jul 16 04:44:04.082087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.081980 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2frgf\" (UniqueName: \"kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf\") pod \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " Jul 16 04:44:04.082087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.081998 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util\") pod \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\" (UID: \"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912\") " Jul 16 04:44:04.082430 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.082398 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle" (OuterVolumeSpecName: "bundle") pod "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" (UID: "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:44:04.084104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.084078 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf" (OuterVolumeSpecName: "kube-api-access-2frgf") pod "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" (UID: "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912"). InnerVolumeSpecName "kube-api-access-2frgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:44:04.087229 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.087207 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util" (OuterVolumeSpecName: "util") pod "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" (UID: "ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:44:04.182965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.182928 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2frgf\" (UniqueName: \"kubernetes.io/projected/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-kube-api-access-2frgf\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:04.182965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.182960 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:04.182965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.182969 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:04.851836 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.851804 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" event={"ID":"ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912","Type":"ContainerDied","Data":"ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900"} Jul 16 04:44:04.852010 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.851851 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddb7181ee3444864aff1c229396f909b459a16251622b113c274fcd52c033900" Jul 16 04:44:04.852010 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:04.851817 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwvpnf" Jul 16 04:44:07.843661 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:07.843632 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-g2f4h" Jul 16 04:44:08.229679 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229644 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9gvx"] Jul 16 04:44:08.229956 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229944 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="pull" Jul 16 04:44:08.230018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229958 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="pull" Jul 16 04:44:08.230018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229969 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="util" Jul 16 04:44:08.230018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229974 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="util" Jul 16 04:44:08.230018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229987 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="extract" Jul 16 04:44:08.230018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.229993 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="extract" Jul 16 04:44:08.230182 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.230048 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb38b58-b6d4-43dc-8fb7-5bfe1e7f7912" containerName="extract" Jul 16 04:44:08.233896 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.233878 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.237744 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.237722 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jfqfk\"" Jul 16 04:44:08.282210 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.282172 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9gvx"] Jul 16 04:44:08.317660 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.317621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfd4\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-kube-api-access-8cfd4\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.317660 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.317662 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.418991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.418951 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfd4\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-kube-api-access-8cfd4\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.418991 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.418993 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.429091 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.429063 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfd4\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-kube-api-access-8cfd4\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.429091 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.429079 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6438860c-b365-40dd-bf5e-1cffe7445a80-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9gvx\" (UID: \"6438860c-b365-40dd-bf5e-1cffe7445a80\") " pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.542635 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.542547 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9gvx" Jul 16 04:44:08.705726 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.705699 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9gvx"] Jul 16 04:44:08.707253 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:08.707211 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6438860c_b365_40dd_bf5e_1cffe7445a80.slice/crio-42fe0335994531540847b8a00d8d9808d5092f9a43b64a7e9ffb7baf78cff2c4 WatchSource:0}: Error finding container 42fe0335994531540847b8a00d8d9808d5092f9a43b64a7e9ffb7baf78cff2c4: Status 404 returned error can't find the container with id 42fe0335994531540847b8a00d8d9808d5092f9a43b64a7e9ffb7baf78cff2c4 Jul 16 04:44:08.872301 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.872266 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9gvx" event={"ID":"6438860c-b365-40dd-bf5e-1cffe7445a80","Type":"ContainerStarted","Data":"b20442abaf1038fe28e50d07b55ec18863ae00594ba0cc51d50514d6b388f213"} Jul 16 04:44:08.872674 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.872309 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9gvx" event={"ID":"6438860c-b365-40dd-bf5e-1cffe7445a80","Type":"ContainerStarted","Data":"42fe0335994531540847b8a00d8d9808d5092f9a43b64a7e9ffb7baf78cff2c4"} Jul 16 04:44:08.892868 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:08.892815 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j9gvx" podStartSLOduration=0.892797132 podStartE2EDuration="892.797132ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:08.892040048 +0000 UTC m=+312.618410348" watchObservedRunningTime="2026-07-16 04:44:08.892797132 +0000 UTC m=+312.619167430" Jul 16 04:44:15.275969 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.275934 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g"] Jul 16 04:44:15.321389 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.321357 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g"] Jul 16 04:44:15.321557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.321487 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.324360 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.324338 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 04:44:15.324600 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.324586 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 04:44:15.325364 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.325351 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-w5jxj\"" Jul 16 04:44:15.373952 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.373916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhgtk\" (UniqueName: \"kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.374118 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.373968 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.374118 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.374052 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.475458 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.475422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.475627 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.475468 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.475627 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.475515 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhgtk\" (UniqueName: \"kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.475906 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.475882 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.475946 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.475891 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.485601 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.485568 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhgtk\" (UniqueName: \"kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.631297 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.631265 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:15.783989 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.783944 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g"] Jul 16 04:44:15.787727 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:15.787699 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0163b33d_6678_4582_838b_03001ad1f3a5.slice/crio-4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081 WatchSource:0}: Error finding container 4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081: Status 404 returned error can't find the container with id 4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081 Jul 16 04:44:15.904243 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:15.904214 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" event={"ID":"0163b33d-6678-4582-838b-03001ad1f3a5","Type":"ContainerStarted","Data":"4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081"} Jul 16 04:44:16.910016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:16.909985 2573 generic.go:356] "Generic (PLEG): container finished" podID="0163b33d-6678-4582-838b-03001ad1f3a5" containerID="abe2222dfcedd37b257bacd462d66cc149b26c6643e67923d2c7070c31a84931" exitCode=0 Jul 16 04:44:16.910430 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:16.910048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" event={"ID":"0163b33d-6678-4582-838b-03001ad1f3a5","Type":"ContainerDied","Data":"abe2222dfcedd37b257bacd462d66cc149b26c6643e67923d2c7070c31a84931"} Jul 16 04:44:16.910540 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:16.910525 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:44:18.920980 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:18.920949 2573 generic.go:356] "Generic (PLEG): container finished" podID="0163b33d-6678-4582-838b-03001ad1f3a5" containerID="665d2e128f2e8e18d4a5451439eab3d993891d59dde85cb70c1107e3b1ecf698" exitCode=0 Jul 16 04:44:18.921377 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:18.921031 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" event={"ID":"0163b33d-6678-4582-838b-03001ad1f3a5","Type":"ContainerDied","Data":"665d2e128f2e8e18d4a5451439eab3d993891d59dde85cb70c1107e3b1ecf698"} Jul 16 04:44:19.928725 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:19.928694 2573 generic.go:356] "Generic (PLEG): container finished" podID="0163b33d-6678-4582-838b-03001ad1f3a5" containerID="5ffdf2ecbfde38d51b5446a14d3d3fee9e7f1e3068eac133231df65358ebf229" exitCode=0 Jul 16 04:44:19.929108 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:19.928752 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" event={"ID":"0163b33d-6678-4582-838b-03001ad1f3a5","Type":"ContainerDied","Data":"5ffdf2ecbfde38d51b5446a14d3d3fee9e7f1e3068eac133231df65358ebf229"} Jul 16 04:44:21.053374 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.053348 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:21.123672 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.123641 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle\") pod \"0163b33d-6678-4582-838b-03001ad1f3a5\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " Jul 16 04:44:21.123818 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.123684 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util\") pod \"0163b33d-6678-4582-838b-03001ad1f3a5\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " Jul 16 04:44:21.123818 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.123775 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhgtk\" (UniqueName: \"kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk\") pod \"0163b33d-6678-4582-838b-03001ad1f3a5\" (UID: \"0163b33d-6678-4582-838b-03001ad1f3a5\") " Jul 16 04:44:21.124543 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.124505 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle" (OuterVolumeSpecName: "bundle") pod "0163b33d-6678-4582-838b-03001ad1f3a5" (UID: "0163b33d-6678-4582-838b-03001ad1f3a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:44:21.125858 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.125829 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk" (OuterVolumeSpecName: "kube-api-access-fhgtk") pod "0163b33d-6678-4582-838b-03001ad1f3a5" (UID: "0163b33d-6678-4582-838b-03001ad1f3a5"). InnerVolumeSpecName "kube-api-access-fhgtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:44:21.129892 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.129866 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util" (OuterVolumeSpecName: "util") pod "0163b33d-6678-4582-838b-03001ad1f3a5" (UID: "0163b33d-6678-4582-838b-03001ad1f3a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:44:21.225194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.225124 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhgtk\" (UniqueName: \"kubernetes.io/projected/0163b33d-6678-4582-838b-03001ad1f3a5-kube-api-access-fhgtk\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:21.225194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.225163 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:21.225194 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.225174 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0163b33d-6678-4582-838b-03001ad1f3a5-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:44:21.939002 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.938974 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" Jul 16 04:44:21.939002 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.938982 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4qt7g" event={"ID":"0163b33d-6678-4582-838b-03001ad1f3a5","Type":"ContainerDied","Data":"4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081"} Jul 16 04:44:21.939260 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:21.939023 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b0d0aa1ff570840f0eff32343d3a24efb18c0e561fa6d2544d1c0f348dfe081" Jul 16 04:44:31.407427 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407388 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn"] Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407704 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="pull" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407716 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="pull" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407722 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="extract" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407727 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="extract" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407739 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="util" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407744 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="util" Jul 16 04:44:31.407880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.407809 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0163b33d-6678-4582-838b-03001ad1f3a5" containerName="extract" Jul 16 04:44:31.410909 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.410892 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.414617 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.414593 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 04:44:31.414617 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.414612 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 04:44:31.414801 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.414667 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 04:44:31.415275 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.415257 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x5n9f\"" Jul 16 04:44:31.415275 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.415272 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 04:44:31.439636 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.439603 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn"] Jul 16 04:44:31.514638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.514604 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.514638 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.514642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.514848 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.514713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.514848 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.514749 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr4pl\" (UniqueName: \"kubernetes.io/projected/1a2d8752-4130-44f7-bd13-e3d3867532d2-kube-api-access-wr4pl\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.514848 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.514788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616048 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616048 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616051 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr4pl\" (UniqueName: \"kubernetes.io/projected/1a2d8752-4130-44f7-bd13-e3d3867532d2-kube-api-access-wr4pl\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616338 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616338 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616170 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616338 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.616570 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.616538 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-manifests\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.618299 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.618265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a2d8752-4130-44f7-bd13-e3d3867532d2-tmp\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.618555 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.618537 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.618614 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.618538 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a2d8752-4130-44f7-bd13-e3d3867532d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.626380 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.626356 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr4pl\" (UniqueName: \"kubernetes.io/projected/1a2d8752-4130-44f7-bd13-e3d3867532d2-kube-api-access-wr4pl\") pod \"opendatahub-operator-controller-manager-7b4b7cb559-nj4sn\" (UID: \"1a2d8752-4130-44f7-bd13-e3d3867532d2\") " pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.721238 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.721133 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:31.878639 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.878612 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn"] Jul 16 04:44:31.881266 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:31.881230 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2d8752_4130_44f7_bd13_e3d3867532d2.slice/crio-10ecc5119f496f7bea0d693062aa82a2478555adcd0f68b90bf1110cb6451aa8 WatchSource:0}: Error finding container 10ecc5119f496f7bea0d693062aa82a2478555adcd0f68b90bf1110cb6451aa8: Status 404 returned error can't find the container with id 10ecc5119f496f7bea0d693062aa82a2478555adcd0f68b90bf1110cb6451aa8 Jul 16 04:44:31.985225 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:31.985128 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" event={"ID":"1a2d8752-4130-44f7-bd13-e3d3867532d2","Type":"ContainerStarted","Data":"10ecc5119f496f7bea0d693062aa82a2478555adcd0f68b90bf1110cb6451aa8"} Jul 16 04:44:36.010116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:36.010082 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" event={"ID":"1a2d8752-4130-44f7-bd13-e3d3867532d2","Type":"ContainerStarted","Data":"b3f6b82c568d53e045204a459c6f8fa0220792994968e811b2dec04570a67a92"} Jul 16 04:44:37.015607 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:37.015568 2573 generic.go:356] "Generic (PLEG): container finished" podID="1a2d8752-4130-44f7-bd13-e3d3867532d2" containerID="b3f6b82c568d53e045204a459c6f8fa0220792994968e811b2dec04570a67a92" exitCode=0 Jul 16 04:44:37.016049 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:37.015653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" event={"ID":"1a2d8752-4130-44f7-bd13-e3d3867532d2","Type":"ContainerDied","Data":"b3f6b82c568d53e045204a459c6f8fa0220792994968e811b2dec04570a67a92"} Jul 16 04:44:38.022422 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:38.022386 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" event={"ID":"1a2d8752-4130-44f7-bd13-e3d3867532d2","Type":"ContainerStarted","Data":"8f2b3fa6eae2a37964de62b7c434395712d3ddc893cfd92b04438965c9ca12d0"} Jul 16 04:44:38.022810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:38.022536 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:38.046735 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:38.046688 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" podStartSLOduration=2.449130935 podStartE2EDuration="7.046672722s" totalImagesPullingTime="3.511913605s" totalInitContainerRuntime="1.085628182s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:38.045047578 +0000 UTC m=+341.771417878" watchObservedRunningTime="2026-07-16 04:44:38.046672722 +0000 UTC m=+341.773043020" Jul 16 04:44:49.029210 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:49.029175 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b4b7cb559-nj4sn" Jul 16 04:44:51.600061 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.600027 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-dcxmr"] Jul 16 04:44:51.606203 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.606171 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.610419 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.610395 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 16 04:44:51.611068 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.611030 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 16 04:44:51.611296 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.611275 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-84b2h\"" Jul 16 04:44:51.611401 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.611309 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 04:44:51.611401 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.611383 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 04:44:51.630263 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.630235 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-dcxmr"] Jul 16 04:44:51.685821 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.685773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkfmk\" (UniqueName: \"kubernetes.io/projected/4349e764-9aa9-4fb3-b719-1eeb64a25970-kube-api-access-qkfmk\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.685984 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.685852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4349e764-9aa9-4fb3-b719-1eeb64a25970-tls-certs\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.685984 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.685878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4349e764-9aa9-4fb3-b719-1eeb64a25970-tmp\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.786616 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.786576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkfmk\" (UniqueName: \"kubernetes.io/projected/4349e764-9aa9-4fb3-b719-1eeb64a25970-kube-api-access-qkfmk\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.786786 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.786631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4349e764-9aa9-4fb3-b719-1eeb64a25970-tls-certs\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.786786 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.786649 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4349e764-9aa9-4fb3-b719-1eeb64a25970-tmp\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.788913 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.788880 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4349e764-9aa9-4fb3-b719-1eeb64a25970-tmp\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.789095 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.789077 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4349e764-9aa9-4fb3-b719-1eeb64a25970-tls-certs\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.813224 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.813195 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkfmk\" (UniqueName: \"kubernetes.io/projected/4349e764-9aa9-4fb3-b719-1eeb64a25970-kube-api-access-qkfmk\") pod \"kube-auth-proxy-68764b5749-dcxmr\" (UID: \"4349e764-9aa9-4fb3-b719-1eeb64a25970\") " pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:51.915714 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:51.915684 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" Jul 16 04:44:52.080779 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:52.080753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68764b5749-dcxmr"] Jul 16 04:44:52.083315 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:52.083290 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4349e764_9aa9_4fb3_b719_1eeb64a25970.slice/crio-f0feca41e3e882f001a4b1a5e67f402d3fc5d24af1e579285c86044595c9ef12 WatchSource:0}: Error finding container f0feca41e3e882f001a4b1a5e67f402d3fc5d24af1e579285c86044595c9ef12: Status 404 returned error can't find the container with id f0feca41e3e882f001a4b1a5e67f402d3fc5d24af1e579285c86044595c9ef12 Jul 16 04:44:53.094504 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:53.094464 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" event={"ID":"4349e764-9aa9-4fb3-b719-1eeb64a25970","Type":"ContainerStarted","Data":"f0feca41e3e882f001a4b1a5e67f402d3fc5d24af1e579285c86044595c9ef12"} Jul 16 04:44:55.106193 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:55.106139 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" event={"ID":"4349e764-9aa9-4fb3-b719-1eeb64a25970","Type":"ContainerStarted","Data":"56a7a0299cbba899ebea46a11b411eb4caf3e024f883370a52ce135c385061c2"} Jul 16 04:44:55.134559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:55.134506 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68764b5749-dcxmr" podStartSLOduration=1.38829325 podStartE2EDuration="4.13448755s" totalImagesPullingTime="2.7461943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:44:55.132965466 +0000 UTC m=+358.859335765" watchObservedRunningTime="2026-07-16 04:44:55.13448755 +0000 UTC m=+358.860857852" Jul 16 04:44:59.492013 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.491974 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz"] Jul 16 04:44:59.496911 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.496892 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.500741 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500717 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 04:44:59.500898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500752 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 04:44:59.500898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500766 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 04:44:59.500898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500721 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vhszm\"" Jul 16 04:44:59.500898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500823 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 04:44:59.500898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.500723 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 04:44:59.507691 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.507665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz"] Jul 16 04:44:59.557523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.557483 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.557681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.557585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-metrics-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.557681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.557638 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0b2c0d7-cd10-4e46-b679-58bb1881b202-manager-config\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.557681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.557661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p2mv\" (UniqueName: \"kubernetes.io/projected/f0b2c0d7-cd10-4e46-b679-58bb1881b202-kube-api-access-2p2mv\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.658938 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.658899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.659117 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.658971 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-metrics-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.659117 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.658993 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0b2c0d7-cd10-4e46-b679-58bb1881b202-manager-config\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.659117 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.659016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p2mv\" (UniqueName: \"kubernetes.io/projected/f0b2c0d7-cd10-4e46-b679-58bb1881b202-kube-api-access-2p2mv\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.659742 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.659713 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f0b2c0d7-cd10-4e46-b679-58bb1881b202-manager-config\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.661608 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.661576 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-metrics-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.661608 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.661592 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0b2c0d7-cd10-4e46-b679-58bb1881b202-cert\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.671760 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.671730 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p2mv\" (UniqueName: \"kubernetes.io/projected/f0b2c0d7-cd10-4e46-b679-58bb1881b202-kube-api-access-2p2mv\") pod \"lws-controller-manager-546f8cbbbf-zxcbz\" (UID: \"f0b2c0d7-cd10-4e46-b679-58bb1881b202\") " pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.806778 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.806687 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:44:59.939773 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:44:59.939749 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz"] Jul 16 04:44:59.941530 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:44:59.941497 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b2c0d7_cd10_4e46_b679_58bb1881b202.slice/crio-2cfeb178e8047068f0bf21b18ce9e401c8e788236f90414af327a9d17d900dc7 WatchSource:0}: Error finding container 2cfeb178e8047068f0bf21b18ce9e401c8e788236f90414af327a9d17d900dc7: Status 404 returned error can't find the container with id 2cfeb178e8047068f0bf21b18ce9e401c8e788236f90414af327a9d17d900dc7 Jul 16 04:45:00.131232 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:45:00.131193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" event={"ID":"f0b2c0d7-cd10-4e46-b679-58bb1881b202","Type":"ContainerStarted","Data":"2cfeb178e8047068f0bf21b18ce9e401c8e788236f90414af327a9d17d900dc7"} Jul 16 04:45:02.143666 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:45:02.143625 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" event={"ID":"f0b2c0d7-cd10-4e46-b679-58bb1881b202","Type":"ContainerStarted","Data":"8af77b6a486a43b8746a978dc6af5a443f98b71a6bd7e047c18bcb6831ded2b1"} Jul 16 04:45:02.144052 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:45:02.143702 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:45:02.210461 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:45:02.210396 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" podStartSLOduration=1.5252749460000001 podStartE2EDuration="3.210373511s" totalImagesPullingTime="1.685098565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:44:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:45:02.209777102 +0000 UTC m=+365.936147422" watchObservedRunningTime="2026-07-16 04:45:02.210373511 +0000 UTC m=+365.936743811" Jul 16 04:45:13.150572 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:45:13.150540 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-546f8cbbbf-zxcbz" Jul 16 04:46:07.662291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.662256 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2"] Jul 16 04:46:07.665989 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.665962 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.669240 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.669212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 04:46:07.669403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.669387 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 04:46:07.670247 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.670229 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8r5pn\"" Jul 16 04:46:07.680375 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.680354 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2"] Jul 16 04:46:07.731528 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.731495 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.731528 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.731533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.731753 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.731608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8xwf\" (UniqueName: \"kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.832965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.832915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.832965 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.832972 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.833206 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.833176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8xwf\" (UniqueName: \"kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.833361 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.833347 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.833402 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.833372 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.843487 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.843452 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8xwf\" (UniqueName: \"kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:07.975601 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:07.975517 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:08.113531 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.113496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2"] Jul 16 04:46:08.115266 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:08.115237 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48054b49_7800_43a6_ab7e_0eec5a76f2c2.slice/crio-b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261 WatchSource:0}: Error finding container b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261: Status 404 returned error can't find the container with id b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261 Jul 16 04:46:08.454867 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.454837 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg"] Jul 16 04:46:08.456016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.455987 2573 generic.go:356] "Generic (PLEG): container finished" podID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerID="2b6bd956063a35b2e5f7d1d2d38febc7b4676548d37f4ff7ee3e6328b3988336" exitCode=0 Jul 16 04:46:08.458467 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.458444 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" event={"ID":"48054b49-7800-43a6-ab7e-0eec5a76f2c2","Type":"ContainerDied","Data":"2b6bd956063a35b2e5f7d1d2d38febc7b4676548d37f4ff7ee3e6328b3988336"} Jul 16 04:46:08.458570 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.458476 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" event={"ID":"48054b49-7800-43a6-ab7e-0eec5a76f2c2","Type":"ContainerStarted","Data":"b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261"} Jul 16 04:46:08.458570 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.458551 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.466561 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.466539 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg"] Jul 16 04:46:08.538736 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.538690 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.538932 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.538750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.538932 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.538863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44szh\" (UniqueName: \"kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.639491 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.639445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.639687 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.639531 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44szh\" (UniqueName: \"kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.639687 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.639643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.639865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.639838 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.639947 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.639927 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.649159 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.649124 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44szh\" (UniqueName: \"kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.768683 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.768593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:08.852842 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.852803 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj"] Jul 16 04:46:08.857634 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.857611 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:08.865964 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.865708 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj"] Jul 16 04:46:08.915493 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.915461 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg"] Jul 16 04:46:08.918615 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:08.918587 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295dfdc2_b7cd_4fff_b663_69392aa1fac1.slice/crio-a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be WatchSource:0}: Error finding container a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be: Status 404 returned error can't find the container with id a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be Jul 16 04:46:08.943063 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.943034 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rnh4\" (UniqueName: \"kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:08.943187 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.943101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:08.943187 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:08.943135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.044012 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.043974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rnh4\" (UniqueName: \"kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.044190 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.044083 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.044190 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.044133 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.044564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.044541 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.044658 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.044611 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.053386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.053365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rnh4\" (UniqueName: \"kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.170395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.170359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:09.255414 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.255235 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87"] Jul 16 04:46:09.260291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.260262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.268132 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.268106 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87"] Jul 16 04:46:09.314865 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.314838 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj"] Jul 16 04:46:09.315724 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:09.315701 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465b14e0_d9a4_400a_8080_d039f504a226.slice/crio-691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708 WatchSource:0}: Error finding container 691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708: Status 404 returned error can't find the container with id 691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708 Jul 16 04:46:09.347066 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.347043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6c6n\" (UniqueName: \"kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.347178 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.347087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.347178 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.347121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.447902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.447857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6c6n\" (UniqueName: \"kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.447902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.447908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.448156 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.447938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.448277 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.448254 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.448333 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.448317 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.457442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.457421 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6c6n\" (UniqueName: \"kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.462664 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.462627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerStarted","Data":"6bac02c785cb40a76df93ee5f3513583464af7a97ec715acb46bd8b2fe8976d2"} Jul 16 04:46:09.462802 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.462676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerStarted","Data":"691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708"} Jul 16 04:46:09.464214 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.464192 2573 generic.go:356] "Generic (PLEG): container finished" podID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerID="070cdc77f9c4033fd2439fa2e1d820f99185ce88ddbc291707f079ef6eaffcf1" exitCode=0 Jul 16 04:46:09.464325 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.464280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" event={"ID":"295dfdc2-b7cd-4fff-b663-69392aa1fac1","Type":"ContainerDied","Data":"070cdc77f9c4033fd2439fa2e1d820f99185ce88ddbc291707f079ef6eaffcf1"} Jul 16 04:46:09.464325 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.464324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" event={"ID":"295dfdc2-b7cd-4fff-b663-69392aa1fac1","Type":"ContainerStarted","Data":"a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be"} Jul 16 04:46:09.466054 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.466032 2573 generic.go:356] "Generic (PLEG): container finished" podID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerID="8292043522fdd05b8198185acff20f141389f378a3c444bbe6b0dcc3028cb074" exitCode=0 Jul 16 04:46:09.466188 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.466167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" event={"ID":"48054b49-7800-43a6-ab7e-0eec5a76f2c2","Type":"ContainerDied","Data":"8292043522fdd05b8198185acff20f141389f378a3c444bbe6b0dcc3028cb074"} Jul 16 04:46:09.592484 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.592455 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:09.725234 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:09.725207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87"] Jul 16 04:46:09.726914 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:09.726886 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070c9160_66f4_4924_ba24_06001e31b5fa.slice/crio-15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564 WatchSource:0}: Error finding container 15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564: Status 404 returned error can't find the container with id 15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564 Jul 16 04:46:10.473212 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.473108 2573 generic.go:356] "Generic (PLEG): container finished" podID="465b14e0-d9a4-400a-8080-d039f504a226" containerID="6bac02c785cb40a76df93ee5f3513583464af7a97ec715acb46bd8b2fe8976d2" exitCode=0 Jul 16 04:46:10.473212 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.473189 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerDied","Data":"6bac02c785cb40a76df93ee5f3513583464af7a97ec715acb46bd8b2fe8976d2"} Jul 16 04:46:10.475224 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.475197 2573 generic.go:356] "Generic (PLEG): container finished" podID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerID="f2ad8f4b100eb985368dfd95b2e449d5cf213a9ae2360216d73abca54b59ebd5" exitCode=0 Jul 16 04:46:10.475408 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.475240 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" event={"ID":"295dfdc2-b7cd-4fff-b663-69392aa1fac1","Type":"ContainerDied","Data":"f2ad8f4b100eb985368dfd95b2e449d5cf213a9ae2360216d73abca54b59ebd5"} Jul 16 04:46:10.483657 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.483578 2573 generic.go:356] "Generic (PLEG): container finished" podID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerID="5ff8643173006e85b3d557049ee92518144a1df1ca06aa4507afc8b9316dc12b" exitCode=0 Jul 16 04:46:10.483751 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.483716 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" event={"ID":"48054b49-7800-43a6-ab7e-0eec5a76f2c2","Type":"ContainerDied","Data":"5ff8643173006e85b3d557049ee92518144a1df1ca06aa4507afc8b9316dc12b"} Jul 16 04:46:10.485514 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.485485 2573 generic.go:356] "Generic (PLEG): container finished" podID="070c9160-66f4-4924-ba24-06001e31b5fa" containerID="097e8d1eb8e0b00d1564f132b7e76c80c05513e5e913d583b122b40b68113d0d" exitCode=0 Jul 16 04:46:10.485618 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.485558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" event={"ID":"070c9160-66f4-4924-ba24-06001e31b5fa","Type":"ContainerDied","Data":"097e8d1eb8e0b00d1564f132b7e76c80c05513e5e913d583b122b40b68113d0d"} Jul 16 04:46:10.485618 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:10.485590 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" event={"ID":"070c9160-66f4-4924-ba24-06001e31b5fa","Type":"ContainerStarted","Data":"15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564"} Jul 16 04:46:11.494285 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.494261 2573 generic.go:356] "Generic (PLEG): container finished" podID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerID="16a52cbd26e7a1827fd4139c47d7ff1a096367d677007d2509c89e23085968e7" exitCode=0 Jul 16 04:46:11.494619 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.494337 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" event={"ID":"295dfdc2-b7cd-4fff-b663-69392aa1fac1","Type":"ContainerDied","Data":"16a52cbd26e7a1827fd4139c47d7ff1a096367d677007d2509c89e23085968e7"} Jul 16 04:46:11.496131 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.496111 2573 generic.go:356] "Generic (PLEG): container finished" podID="070c9160-66f4-4924-ba24-06001e31b5fa" containerID="59bcc611461e20fe17ab340a5b20e802a2b74be65a6ce6f101d23587d91b89ce" exitCode=0 Jul 16 04:46:11.496231 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.496175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" event={"ID":"070c9160-66f4-4924-ba24-06001e31b5fa","Type":"ContainerDied","Data":"59bcc611461e20fe17ab340a5b20e802a2b74be65a6ce6f101d23587d91b89ce"} Jul 16 04:46:11.624011 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.623988 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:11.770969 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.770932 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8xwf\" (UniqueName: \"kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf\") pod \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " Jul 16 04:46:11.771132 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.770997 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util\") pod \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " Jul 16 04:46:11.771132 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.771013 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle\") pod \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\" (UID: \"48054b49-7800-43a6-ab7e-0eec5a76f2c2\") " Jul 16 04:46:11.771555 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.771528 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle" (OuterVolumeSpecName: "bundle") pod "48054b49-7800-43a6-ab7e-0eec5a76f2c2" (UID: "48054b49-7800-43a6-ab7e-0eec5a76f2c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:11.773076 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.773053 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf" (OuterVolumeSpecName: "kube-api-access-w8xwf") pod "48054b49-7800-43a6-ab7e-0eec5a76f2c2" (UID: "48054b49-7800-43a6-ab7e-0eec5a76f2c2"). InnerVolumeSpecName "kube-api-access-w8xwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:11.776239 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.776206 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util" (OuterVolumeSpecName: "util") pod "48054b49-7800-43a6-ab7e-0eec5a76f2c2" (UID: "48054b49-7800-43a6-ab7e-0eec5a76f2c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:11.872319 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.872290 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8xwf\" (UniqueName: \"kubernetes.io/projected/48054b49-7800-43a6-ab7e-0eec5a76f2c2-kube-api-access-w8xwf\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:11.872319 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.872315 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:11.872319 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:11.872326 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/48054b49-7800-43a6-ab7e-0eec5a76f2c2-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:12.503179 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.503076 2573 generic.go:356] "Generic (PLEG): container finished" podID="465b14e0-d9a4-400a-8080-d039f504a226" containerID="855ffcac6b1c63c6e77dd9f3847478d5f23d0dd49466af6aa6dfb97d7946021b" exitCode=0 Jul 16 04:46:12.503603 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.503175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerDied","Data":"855ffcac6b1c63c6e77dd9f3847478d5f23d0dd49466af6aa6dfb97d7946021b"} Jul 16 04:46:12.505131 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.505116 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" Jul 16 04:46:12.505242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.505126 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2" event={"ID":"48054b49-7800-43a6-ab7e-0eec5a76f2c2","Type":"ContainerDied","Data":"b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261"} Jul 16 04:46:12.505242 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.505185 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b70732f00cff1309cd053693148e852698be985371eff16172fbae482ad28261" Jul 16 04:46:12.507279 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.507257 2573 generic.go:356] "Generic (PLEG): container finished" podID="070c9160-66f4-4924-ba24-06001e31b5fa" containerID="bb4e6296e383853bd838ee80b49dac599c4f252fb840310bab176d8a1deaf1e2" exitCode=0 Jul 16 04:46:12.507388 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.507342 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" event={"ID":"070c9160-66f4-4924-ba24-06001e31b5fa","Type":"ContainerDied","Data":"bb4e6296e383853bd838ee80b49dac599c4f252fb840310bab176d8a1deaf1e2"} Jul 16 04:46:12.643361 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.643334 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:12.780810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.780709 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util\") pod \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " Jul 16 04:46:12.780810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.780780 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44szh\" (UniqueName: \"kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh\") pod \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " Jul 16 04:46:12.780810 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.780809 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle\") pod \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\" (UID: \"295dfdc2-b7cd-4fff-b663-69392aa1fac1\") " Jul 16 04:46:12.781472 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.781443 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle" (OuterVolumeSpecName: "bundle") pod "295dfdc2-b7cd-4fff-b663-69392aa1fac1" (UID: "295dfdc2-b7cd-4fff-b663-69392aa1fac1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:12.782908 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.782889 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh" (OuterVolumeSpecName: "kube-api-access-44szh") pod "295dfdc2-b7cd-4fff-b663-69392aa1fac1" (UID: "295dfdc2-b7cd-4fff-b663-69392aa1fac1"). InnerVolumeSpecName "kube-api-access-44szh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:12.786171 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.786134 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util" (OuterVolumeSpecName: "util") pod "295dfdc2-b7cd-4fff-b663-69392aa1fac1" (UID: "295dfdc2-b7cd-4fff-b663-69392aa1fac1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:12.882074 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.882042 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:12.882074 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.882072 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44szh\" (UniqueName: \"kubernetes.io/projected/295dfdc2-b7cd-4fff-b663-69392aa1fac1-kube-api-access-44szh\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:12.882074 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:12.882082 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/295dfdc2-b7cd-4fff-b663-69392aa1fac1-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:13.514167 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.514058 2573 generic.go:356] "Generic (PLEG): container finished" podID="465b14e0-d9a4-400a-8080-d039f504a226" containerID="71e171291a876f915bc6d3336dc85448ca526fb6bfbb877174e0e45603704438" exitCode=0 Jul 16 04:46:13.514167 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.514100 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerDied","Data":"71e171291a876f915bc6d3336dc85448ca526fb6bfbb877174e0e45603704438"} Jul 16 04:46:13.515913 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.515897 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" Jul 16 04:46:13.516022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.515928 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg" event={"ID":"295dfdc2-b7cd-4fff-b663-69392aa1fac1","Type":"ContainerDied","Data":"a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be"} Jul 16 04:46:13.516022 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.515960 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7c50835851ded8b2682092a8ff1907bc9fb31b3f32bf69569427684bc9153be" Jul 16 04:46:13.650177 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.649365 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:13.791502 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.791413 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle\") pod \"070c9160-66f4-4924-ba24-06001e31b5fa\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " Jul 16 04:46:13.791654 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.791514 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util\") pod \"070c9160-66f4-4924-ba24-06001e31b5fa\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " Jul 16 04:46:13.791654 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.791598 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6c6n\" (UniqueName: \"kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n\") pod \"070c9160-66f4-4924-ba24-06001e31b5fa\" (UID: \"070c9160-66f4-4924-ba24-06001e31b5fa\") " Jul 16 04:46:13.791952 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.791931 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle" (OuterVolumeSpecName: "bundle") pod "070c9160-66f4-4924-ba24-06001e31b5fa" (UID: "070c9160-66f4-4924-ba24-06001e31b5fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:13.793656 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.793634 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n" (OuterVolumeSpecName: "kube-api-access-x6c6n") pod "070c9160-66f4-4924-ba24-06001e31b5fa" (UID: "070c9160-66f4-4924-ba24-06001e31b5fa"). InnerVolumeSpecName "kube-api-access-x6c6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:13.796881 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.796845 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util" (OuterVolumeSpecName: "util") pod "070c9160-66f4-4924-ba24-06001e31b5fa" (UID: "070c9160-66f4-4924-ba24-06001e31b5fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:13.892254 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.892210 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:13.892254 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.892245 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/070c9160-66f4-4924-ba24-06001e31b5fa-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:13.892254 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:13.892259 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6c6n\" (UniqueName: \"kubernetes.io/projected/070c9160-66f4-4924-ba24-06001e31b5fa-kube-api-access-x6c6n\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:14.523403 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.523371 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" Jul 16 04:46:14.523771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.523401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87" event={"ID":"070c9160-66f4-4924-ba24-06001e31b5fa","Type":"ContainerDied","Data":"15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564"} Jul 16 04:46:14.523771 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.523447 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15597b4a6b36b0f4143e30429df88a25bfae8c412f6635899411cab9e7ce0564" Jul 16 04:46:14.654654 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.654629 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:14.801193 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.801070 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle\") pod \"465b14e0-d9a4-400a-8080-d039f504a226\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " Jul 16 04:46:14.801193 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.801195 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util\") pod \"465b14e0-d9a4-400a-8080-d039f504a226\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " Jul 16 04:46:14.801434 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.801218 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rnh4\" (UniqueName: \"kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4\") pod \"465b14e0-d9a4-400a-8080-d039f504a226\" (UID: \"465b14e0-d9a4-400a-8080-d039f504a226\") " Jul 16 04:46:14.801578 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.801554 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle" (OuterVolumeSpecName: "bundle") pod "465b14e0-d9a4-400a-8080-d039f504a226" (UID: "465b14e0-d9a4-400a-8080-d039f504a226"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:14.803432 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.803409 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4" (OuterVolumeSpecName: "kube-api-access-6rnh4") pod "465b14e0-d9a4-400a-8080-d039f504a226" (UID: "465b14e0-d9a4-400a-8080-d039f504a226"). InnerVolumeSpecName "kube-api-access-6rnh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:14.805842 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.805818 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util" (OuterVolumeSpecName: "util") pod "465b14e0-d9a4-400a-8080-d039f504a226" (UID: "465b14e0-d9a4-400a-8080-d039f504a226"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:46:14.901984 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.901941 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-util\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:14.901984 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.901977 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rnh4\" (UniqueName: \"kubernetes.io/projected/465b14e0-d9a4-400a-8080-d039f504a226-kube-api-access-6rnh4\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:14.901984 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:14.901989 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/465b14e0-d9a4-400a-8080-d039f504a226-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:15.530912 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:15.530886 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" Jul 16 04:46:15.531274 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:15.530915 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj" event={"ID":"465b14e0-d9a4-400a-8080-d039f504a226","Type":"ContainerDied","Data":"691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708"} Jul 16 04:46:15.531274 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:15.530958 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="691e1745ead7d4e8bb40fd071887fa16dc8fb6bcdf985c9e9b0d5cabbf672708" Jul 16 04:46:18.025439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025407 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-566696f4dd-sfrmk"] Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025760 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025772 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025781 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025788 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025797 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025802 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025809 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="pull" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025814 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="pull" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025822 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="pull" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025826 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="pull" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025832 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="util" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025837 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="util" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025843 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="extract" Jul 16 04:46:18.025841 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025849 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="extract" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025860 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="pull" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025864 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="pull" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025869 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="pull" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025874 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="pull" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025886 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025890 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025898 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025903 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025909 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025914 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="util" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025964 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="465b14e0-d9a4-400a-8080-d039f504a226" containerName="extract" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025972 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="070c9160-66f4-4924-ba24-06001e31b5fa" containerName="extract" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025978 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="48054b49-7800-43a6-ab7e-0eec5a76f2c2" containerName="extract" Jul 16 04:46:18.026439 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.025986 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="295dfdc2-b7cd-4fff-b663-69392aa1fac1" containerName="extract" Jul 16 04:46:18.029829 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.029806 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.039893 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.039869 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566696f4dd-sfrmk"] Jul 16 04:46:18.131450 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131413 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131450 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bhv\" (UniqueName: \"kubernetes.io/projected/547e5b15-2c4b-48cf-bc4b-54b8825faef5-kube-api-access-b5bhv\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-trusted-ca-bundle\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-oauth-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131737 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-service-ca\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.131737 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.131684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-oauth-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.232951 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.232910 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-trusted-ca-bundle\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.232951 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.232950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-oauth-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.232974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-service-ca\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.232990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-oauth-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233013 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233198 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bhv\" (UniqueName: \"kubernetes.io/projected/547e5b15-2c4b-48cf-bc4b-54b8825faef5-kube-api-access-b5bhv\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233845 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233813 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-service-ca\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.233908 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233892 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-oauth-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.234011 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.233990 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.234119 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.234095 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/547e5b15-2c4b-48cf-bc4b-54b8825faef5-trusted-ca-bundle\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.235576 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.235554 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-serving-cert\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.235681 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.235596 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/547e5b15-2c4b-48cf-bc4b-54b8825faef5-console-oauth-config\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.241863 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.241823 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bhv\" (UniqueName: \"kubernetes.io/projected/547e5b15-2c4b-48cf-bc4b-54b8825faef5-kube-api-access-b5bhv\") pod \"console-566696f4dd-sfrmk\" (UID: \"547e5b15-2c4b-48cf-bc4b-54b8825faef5\") " pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.339183 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.339081 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:18.471927 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.471901 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566696f4dd-sfrmk"] Jul 16 04:46:18.473630 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:18.473602 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547e5b15_2c4b_48cf_bc4b_54b8825faef5.slice/crio-72af697794391421d6b60656e913def155b96695ddfcf692cce0401393b6414d WatchSource:0}: Error finding container 72af697794391421d6b60656e913def155b96695ddfcf692cce0401393b6414d: Status 404 returned error can't find the container with id 72af697794391421d6b60656e913def155b96695ddfcf692cce0401393b6414d Jul 16 04:46:18.547297 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.547257 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566696f4dd-sfrmk" event={"ID":"547e5b15-2c4b-48cf-bc4b-54b8825faef5","Type":"ContainerStarted","Data":"049a9e504df3db04a82595ac27da4813648717d646384ea7143dde8e596fb8ca"} Jul 16 04:46:18.547455 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.547306 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566696f4dd-sfrmk" event={"ID":"547e5b15-2c4b-48cf-bc4b-54b8825faef5","Type":"ContainerStarted","Data":"72af697794391421d6b60656e913def155b96695ddfcf692cce0401393b6414d"} Jul 16 04:46:18.571381 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:18.571220 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-566696f4dd-sfrmk" podStartSLOduration=0.571199502 podStartE2EDuration="571.199502ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:18.569633723 +0000 UTC m=+442.296004036" watchObservedRunningTime="2026-07-16 04:46:18.571199502 +0000 UTC m=+442.297569800" Jul 16 04:46:28.339653 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:28.339542 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:28.339653 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:28.339628 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:28.345327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:28.345303 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:28.598608 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:28.598530 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-566696f4dd-sfrmk" Jul 16 04:46:28.657296 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:28.657257 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:46:45.903069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.903033 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:46:45.907795 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.907772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:45.911358 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.911338 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgrpj\"" Jul 16 04:46:45.911919 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.911897 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 04:46:45.912062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.912046 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 04:46:45.925485 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.925448 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:46:45.997994 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.997955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42g2\" (UniqueName: \"kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:45.998221 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:45.998049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.099039 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.099000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.099255 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.099167 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42g2\" (UniqueName: \"kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.099453 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.099430 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.109459 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.109423 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42g2\" (UniqueName: \"kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2\") pod \"kuadrant-operator-controller-manager-86878c769f-qt6pm\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.217847 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.217748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:46.357955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.357927 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:46:46.359471 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:46:46.359444 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50481031_d950_4868_bea4_389e987a1440.slice/crio-8b32275f933dd2f8e5ae8e51df2bc02d6e3c50537decd5a565e44f69bdd91a24 WatchSource:0}: Error finding container 8b32275f933dd2f8e5ae8e51df2bc02d6e3c50537decd5a565e44f69bdd91a24: Status 404 returned error can't find the container with id 8b32275f933dd2f8e5ae8e51df2bc02d6e3c50537decd5a565e44f69bdd91a24 Jul 16 04:46:46.689414 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:46.689378 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" event={"ID":"50481031-d950-4868-bea4-389e987a1440","Type":"ContainerStarted","Data":"8b32275f933dd2f8e5ae8e51df2bc02d6e3c50537decd5a565e44f69bdd91a24"} Jul 16 04:46:51.720622 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:51.720578 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" event={"ID":"50481031-d950-4868-bea4-389e987a1440","Type":"ContainerStarted","Data":"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c"} Jul 16 04:46:51.721218 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:51.721194 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:46:51.745789 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:51.745734 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" podStartSLOduration=2.055611331 podStartE2EDuration="6.74571771s" totalImagesPullingTime="4.690106379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:46:51.743383757 +0000 UTC m=+475.469754069" watchObservedRunningTime="2026-07-16 04:46:51.74571771 +0000 UTC m=+475.472088009" Jul 16 04:46:53.677714 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:53.677666 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7649768847-78gtc" podUID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" containerName="console" containerID="cri-o://da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d" gracePeriod=15 Jul 16 04:46:53.923088 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:53.923064 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:46:54.077102 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077007 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077102 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077060 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077102 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077098 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077423 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077116 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077423 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077137 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077423 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077205 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szv9n\" (UniqueName: \"kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077423 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077224 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config\") pod \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\" (UID: \"d0d9713d-8a61-412e-8244-8d193c3f5bf6\") " Jul 16 04:46:54.077702 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077648 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:46:54.077702 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077667 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:46:54.077702 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077688 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config" (OuterVolumeSpecName: "console-config") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:46:54.078016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.077833 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 04:46:54.078094 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.078074 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-trusted-ca-bundle\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.078130 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.078103 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-service-ca\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.078130 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.078118 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.078229 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.078134 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d0d9713d-8a61-412e-8244-8d193c3f5bf6-oauth-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.079281 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.079257 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:46:54.079396 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.079375 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n" (OuterVolumeSpecName: "kube-api-access-szv9n") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "kube-api-access-szv9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:46:54.079604 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.079569 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d0d9713d-8a61-412e-8244-8d193c3f5bf6" (UID: "d0d9713d-8a61-412e-8244-8d193c3f5bf6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:46:54.178995 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.178952 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-serving-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.178995 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.178985 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szv9n\" (UniqueName: \"kubernetes.io/projected/d0d9713d-8a61-412e-8244-8d193c3f5bf6-kube-api-access-szv9n\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.178995 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.178995 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d0d9713d-8a61-412e-8244-8d193c3f5bf6-console-oauth-config\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:46:54.736780 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.736747 2573 generic.go:356] "Generic (PLEG): container finished" podID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" containerID="da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d" exitCode=0 Jul 16 04:46:54.737281 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.736807 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7649768847-78gtc" event={"ID":"d0d9713d-8a61-412e-8244-8d193c3f5bf6","Type":"ContainerDied","Data":"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d"} Jul 16 04:46:54.737281 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.736812 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7649768847-78gtc" Jul 16 04:46:54.737281 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.736849 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7649768847-78gtc" event={"ID":"d0d9713d-8a61-412e-8244-8d193c3f5bf6","Type":"ContainerDied","Data":"b251dc3d26a6891cdaf0fd391f9213cf4283139817e3679f31f0a9fbb3f41826"} Jul 16 04:46:54.737281 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.736869 2573 scope.go:117] "RemoveContainer" containerID="da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d" Jul 16 04:46:54.746032 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.746011 2573 scope.go:117] "RemoveContainer" containerID="da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d" Jul 16 04:46:54.746291 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:46:54.746270 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d\": container with ID starting with da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d not found: ID does not exist" containerID="da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d" Jul 16 04:46:54.746360 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.746299 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d"} err="failed to get container status \"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d\": rpc error: code = NotFound desc = could not find container \"da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d\": container with ID starting with da1b7454c49784a96a009c864c55d5473cf9db82c6e11c704e08349bce57146d not found: ID does not exist" Jul 16 04:46:54.758829 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.758799 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:46:54.762946 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:54.762925 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7649768847-78gtc"] Jul 16 04:46:55.512898 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:46:55.512864 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" path="/var/lib/kubelet/pods/d0d9713d-8a61-412e-8244-8d193c3f5bf6/volumes" Jul 16 04:47:03.732489 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:03.732459 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:47:23.118566 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.118529 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:23.119069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.118928 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" containerName="console" Jul 16 04:47:23.119069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.118944 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" containerName="console" Jul 16 04:47:23.119069 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.119031 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d9713d-8a61-412e-8244-8d193c3f5bf6" containerName="console" Jul 16 04:47:23.121920 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.121900 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:23.124252 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.124226 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2m5jz\"" Jul 16 04:47:23.129410 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.129363 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:23.228425 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.228388 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6vpp\" (UniqueName: \"kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp\") pod \"authorino-58cd955675-l7wpv\" (UID: \"a1af3940-92e3-412c-a02a-b5d0ee5835f1\") " pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:23.329800 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.329763 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6vpp\" (UniqueName: \"kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp\") pod \"authorino-58cd955675-l7wpv\" (UID: \"a1af3940-92e3-412c-a02a-b5d0ee5835f1\") " pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:23.341383 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.341357 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6vpp\" (UniqueName: \"kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp\") pod \"authorino-58cd955675-l7wpv\" (UID: \"a1af3940-92e3-412c-a02a-b5d0ee5835f1\") " pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:23.432234 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.432110 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:23.568672 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.568640 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:23.570679 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:47:23.570648 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1af3940_92e3_412c_a02a_b5d0ee5835f1.slice/crio-207ba67718bd5dd1088bd7a7d8da94c0b11c9dce5064433d0ad00fdf94b291bf WatchSource:0}: Error finding container 207ba67718bd5dd1088bd7a7d8da94c0b11c9dce5064433d0ad00fdf94b291bf: Status 404 returned error can't find the container with id 207ba67718bd5dd1088bd7a7d8da94c0b11c9dce5064433d0ad00fdf94b291bf Jul 16 04:47:23.883314 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:23.883280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-l7wpv" event={"ID":"a1af3940-92e3-412c-a02a-b5d0ee5835f1","Type":"ContainerStarted","Data":"207ba67718bd5dd1088bd7a7d8da94c0b11c9dce5064433d0ad00fdf94b291bf"} Jul 16 04:47:27.906552 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:27.906510 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-l7wpv" event={"ID":"a1af3940-92e3-412c-a02a-b5d0ee5835f1","Type":"ContainerStarted","Data":"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19"} Jul 16 04:47:27.921838 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:27.921786 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-l7wpv" podStartSLOduration=1.582394211 podStartE2EDuration="4.921768579s" totalImagesPullingTime="3.339374368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:47:27.920748061 +0000 UTC m=+511.647118361" watchObservedRunningTime="2026-07-16 04:47:27.921768579 +0000 UTC m=+511.648138913" Jul 16 04:47:28.315490 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:28.315402 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:29.916225 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:29.916174 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-l7wpv" podUID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" containerName="authorino" containerID="cri-o://310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19" gracePeriod=30 Jul 16 04:47:30.163694 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.163664 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:30.292806 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.292702 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6vpp\" (UniqueName: \"kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp\") pod \"a1af3940-92e3-412c-a02a-b5d0ee5835f1\" (UID: \"a1af3940-92e3-412c-a02a-b5d0ee5835f1\") " Jul 16 04:47:30.294871 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.294833 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp" (OuterVolumeSpecName: "kube-api-access-l6vpp") pod "a1af3940-92e3-412c-a02a-b5d0ee5835f1" (UID: "a1af3940-92e3-412c-a02a-b5d0ee5835f1"). InnerVolumeSpecName "kube-api-access-l6vpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:30.394386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.394344 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6vpp\" (UniqueName: \"kubernetes.io/projected/a1af3940-92e3-412c-a02a-b5d0ee5835f1-kube-api-access-l6vpp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:47:30.923008 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.922969 2573 generic.go:356] "Generic (PLEG): container finished" podID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" containerID="310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19" exitCode=0 Jul 16 04:47:30.923008 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.923005 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-l7wpv" Jul 16 04:47:30.923500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.923046 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-l7wpv" event={"ID":"a1af3940-92e3-412c-a02a-b5d0ee5835f1","Type":"ContainerDied","Data":"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19"} Jul 16 04:47:30.923500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.923085 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-l7wpv" event={"ID":"a1af3940-92e3-412c-a02a-b5d0ee5835f1","Type":"ContainerDied","Data":"207ba67718bd5dd1088bd7a7d8da94c0b11c9dce5064433d0ad00fdf94b291bf"} Jul 16 04:47:30.923500 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.923103 2573 scope.go:117] "RemoveContainer" containerID="310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19" Jul 16 04:47:30.932268 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.932251 2573 scope.go:117] "RemoveContainer" containerID="310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19" Jul 16 04:47:30.932534 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:47:30.932517 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19\": container with ID starting with 310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19 not found: ID does not exist" containerID="310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19" Jul 16 04:47:30.932570 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.932543 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19"} err="failed to get container status \"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19\": rpc error: code = NotFound desc = could not find container \"310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19\": container with ID starting with 310eb25518f895321d2572a496922320501b35c4a19f5994a40bfeddf8797b19 not found: ID does not exist" Jul 16 04:47:30.944295 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.944265 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:30.948536 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:30.948510 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-l7wpv"] Jul 16 04:47:31.514382 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:31.514347 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" path="/var/lib/kubelet/pods/a1af3940-92e3-412c-a02a-b5d0ee5835f1/volumes" Jul 16 04:47:55.991028 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.990951 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tqw87"] Jul 16 04:47:55.991557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.991338 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" containerName="authorino" Jul 16 04:47:55.991557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.991353 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" containerName="authorino" Jul 16 04:47:55.991557 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.991423 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1af3940-92e3-412c-a02a-b5d0ee5835f1" containerName="authorino" Jul 16 04:47:55.994492 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.994474 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:55.996977 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:55.996959 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2m5jz\"" Jul 16 04:47:56.000948 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:56.000924 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tqw87"] Jul 16 04:47:56.130092 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:56.130053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqvjr\" (UniqueName: \"kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr\") pod \"authorino-5d4b67c446-tqw87\" (UID: \"ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc\") " pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:56.217747 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:56.217710 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tqw87"] Jul 16 04:47:56.217993 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:47:56.217972 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sqvjr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tqw87" podUID="ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc" Jul 16 04:47:56.230785 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:56.230751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqvjr\" (UniqueName: \"kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr\") pod \"authorino-5d4b67c446-tqw87\" (UID: \"ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc\") " pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:56.239189 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:56.239161 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqvjr\" (UniqueName: \"kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr\") pod \"authorino-5d4b67c446-tqw87\" (UID: \"ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc\") " pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:57.063125 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:57.060168 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:57.070589 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:57.070562 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:57.138204 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:57.138168 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqvjr\" (UniqueName: \"kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr\") pod \"ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc\" (UID: \"ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc\") " Jul 16 04:47:57.140292 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:57.140260 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr" (OuterVolumeSpecName: "kube-api-access-sqvjr") pod "ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc" (UID: "ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc"). InnerVolumeSpecName "kube-api-access-sqvjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:47:57.239413 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:57.239378 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqvjr\" (UniqueName: \"kubernetes.io/projected/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc-kube-api-access-sqvjr\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:47:58.065383 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:58.065351 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tqw87" Jul 16 04:47:58.091105 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:58.091074 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tqw87"] Jul 16 04:47:58.094880 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:58.094853 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tqw87"] Jul 16 04:47:59.513284 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:47:59.513255 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc" path="/var/lib/kubelet/pods/ab56ad61-5b7b-458b-90d7-ef8d1a9ad8cc/volumes" Jul 16 04:48:02.372177 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.372127 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:48:02.375564 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.375544 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.377910 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.377882 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 04:48:02.377910 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.377881 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 04:48:02.378081 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.377951 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6ds6k\"" Jul 16 04:48:02.386946 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.386922 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:48:02.485016 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.484991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.485139 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.485049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhkvd\" (UniqueName: \"kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.586291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.586265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.586384 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.586323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhkvd\" (UniqueName: \"kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.588691 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.588670 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.594524 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.594503 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhkvd\" (UniqueName: \"kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd\") pod \"maas-controller-5b6d84dc7c-j257m\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:02.685774 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:02.685717 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:03.024361 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:03.024338 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:48:03.026399 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:48:03.026361 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3484f60_4294_4821_ac65_b29adb7a2ebc.slice/crio-9f6e73f82a70efadcf9f04c7348195df64e894f5b4704d0012e2635f66dff4e4 WatchSource:0}: Error finding container 9f6e73f82a70efadcf9f04c7348195df64e894f5b4704d0012e2635f66dff4e4: Status 404 returned error can't find the container with id 9f6e73f82a70efadcf9f04c7348195df64e894f5b4704d0012e2635f66dff4e4 Jul 16 04:48:03.091155 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:03.091117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" event={"ID":"e3484f60-4294-4821-ac65-b29adb7a2ebc","Type":"ContainerStarted","Data":"9f6e73f82a70efadcf9f04c7348195df64e894f5b4704d0012e2635f66dff4e4"} Jul 16 04:48:06.110429 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:06.110395 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" event={"ID":"e3484f60-4294-4821-ac65-b29adb7a2ebc","Type":"ContainerStarted","Data":"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888"} Jul 16 04:48:06.110794 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:06.110498 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:06.127703 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:06.127650 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" podStartSLOduration=1.827673264 podStartE2EDuration="4.127633514s" totalImagesPullingTime="2.29996025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:48:06.126039354 +0000 UTC m=+549.852409652" watchObservedRunningTime="2026-07-16 04:48:06.127633514 +0000 UTC m=+549.854003813" Jul 16 04:48:17.118371 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:17.118338 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:48:38.778025 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.777988 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-frht7"] Jul 16 04:48:38.784442 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.784422 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:38.788207 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.788187 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dt4lc\"" Jul 16 04:48:38.789646 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.789626 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 04:48:38.818751 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.818716 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-frht7"] Jul 16 04:48:38.918482 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.918439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g58fq\" (UniqueName: \"kubernetes.io/projected/5262d330-1b50-440e-81bb-fea35169d344-kube-api-access-g58fq\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:38.918654 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:38.918505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5262d330-1b50-440e-81bb-fea35169d344-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.019104 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.019055 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g58fq\" (UniqueName: \"kubernetes.io/projected/5262d330-1b50-440e-81bb-fea35169d344-kube-api-access-g58fq\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.019341 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.019176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5262d330-1b50-440e-81bb-fea35169d344-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.019962 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.019936 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5262d330-1b50-440e-81bb-fea35169d344-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.029913 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.029844 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g58fq\" (UniqueName: \"kubernetes.io/projected/5262d330-1b50-440e-81bb-fea35169d344-kube-api-access-g58fq\") pod \"payload-pre-processing-74584bb9f8-frht7\" (UID: \"5262d330-1b50-440e-81bb-fea35169d344\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.094833 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.094792 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:39.231311 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.231279 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-frht7"] Jul 16 04:48:39.231775 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:48:39.231745 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5262d330_1b50_440e_81bb_fea35169d344.slice/crio-ff791a027f083e7fde748e008d63c0dabf01e07c8ca178f0c15cf5cbcf77c3e5 WatchSource:0}: Error finding container ff791a027f083e7fde748e008d63c0dabf01e07c8ca178f0c15cf5cbcf77c3e5: Status 404 returned error can't find the container with id ff791a027f083e7fde748e008d63c0dabf01e07c8ca178f0c15cf5cbcf77c3e5 Jul 16 04:48:39.285511 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:39.285401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" event={"ID":"5262d330-1b50-440e-81bb-fea35169d344","Type":"ContainerStarted","Data":"ff791a027f083e7fde748e008d63c0dabf01e07c8ca178f0c15cf5cbcf77c3e5"} Jul 16 04:48:41.301938 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:41.301906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" event={"ID":"5262d330-1b50-440e-81bb-fea35169d344","Type":"ContainerStarted","Data":"50ae138ece942b1efa394940d4611d26c9a0d0107f7cb575f45f17357e788d65"} Jul 16 04:48:41.302366 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:41.301963 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:41.317789 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:41.317741 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" podStartSLOduration=1.899863162 podStartE2EDuration="3.317687117s" totalImagesPullingTime="1.417823955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:48:41.317137319 +0000 UTC m=+585.043507620" watchObservedRunningTime="2026-07-16 04:48:41.317687117 +0000 UTC m=+585.044057415" Jul 16 04:48:52.309359 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:52.309323 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-frht7" Jul 16 04:48:56.762938 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:56.762909 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:48:56.769347 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:56.769324 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:48:58.086213 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.086180 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz"] Jul 16 04:48:58.089693 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.089676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.092386 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.092355 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 04:48:58.092539 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.092492 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 04:48:58.092539 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.092531 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q2d59\"" Jul 16 04:48:58.093337 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.093320 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 16 04:48:58.103186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.103163 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz"] Jul 16 04:48:58.179711 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179674 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.179711 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.180018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.180018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.180018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxbfk\" (UniqueName: \"kubernetes.io/projected/9f96c3de-a33c-47f5-9875-58322f9f6c96-kube-api-access-bxbfk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.180018 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.179935 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f96c3de-a33c-47f5-9875-58322f9f6c96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281175 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281104 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbfk\" (UniqueName: \"kubernetes.io/projected/9f96c3de-a33c-47f5-9875-58322f9f6c96-kube-api-access-bxbfk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281368 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f96c3de-a33c-47f5-9875-58322f9f6c96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281368 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281368 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281524 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281581 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281710 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281690 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281806 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281722 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.281866 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.281833 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.283733 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.283713 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f96c3de-a33c-47f5-9875-58322f9f6c96-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.283902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.283885 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f96c3de-a33c-47f5-9875-58322f9f6c96-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.288902 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.288880 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbfk\" (UniqueName: \"kubernetes.io/projected/9f96c3de-a33c-47f5-9875-58322f9f6c96-kube-api-access-bxbfk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz\" (UID: \"9f96c3de-a33c-47f5-9875-58322f9f6c96\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.403437 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.403400 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:48:58.551596 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:58.551572 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz"] Jul 16 04:48:58.553264 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:48:58.553236 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f96c3de_a33c_47f5_9875_58322f9f6c96.slice/crio-be8ddcda80798a651a9c7f5d6552e6abbc3ba83c411794b6d9f5ed3f41835919 WatchSource:0}: Error finding container be8ddcda80798a651a9c7f5d6552e6abbc3ba83c411794b6d9f5ed3f41835919: Status 404 returned error can't find the container with id be8ddcda80798a651a9c7f5d6552e6abbc3ba83c411794b6d9f5ed3f41835919 Jul 16 04:48:59.397432 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:48:59.397390 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" event={"ID":"9f96c3de-a33c-47f5-9875-58322f9f6c96","Type":"ContainerStarted","Data":"be8ddcda80798a651a9c7f5d6552e6abbc3ba83c411794b6d9f5ed3f41835919"} Jul 16 04:49:00.406505 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:00.406461 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" event={"ID":"9f96c3de-a33c-47f5-9875-58322f9f6c96","Type":"ContainerStarted","Data":"7ed718766d3e196354aaa0565e18a591248ec91094e40cac0cbb50ef5b049b78"} Jul 16 04:49:00.429189 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:00.429114 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" podStartSLOduration=1.3184760629999999 podStartE2EDuration="2.429097509s" totalImagesPullingTime="1.110621446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:49:00.426576301 +0000 UTC m=+604.152946601" watchObservedRunningTime="2026-07-16 04:49:00.429097509 +0000 UTC m=+604.155467808" Jul 16 04:49:08.404414 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:08.404365 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:49:08.404899 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:08.404439 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:49:08.416839 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:08.416809 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:49:08.461251 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:08.461214 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz" Jul 16 04:49:29.008112 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.008022 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h"] Jul 16 04:49:29.016941 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.015614 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.019618 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.019590 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 04:49:29.028992 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.028962 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h"] Jul 16 04:49:29.161346 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.161523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.161523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7gt\" (UniqueName: \"kubernetes.io/projected/4f4fc37f-91c3-47c2-b389-2d09a2a76029-kube-api-access-ts7gt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.161523 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161473 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.161660 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.161660 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.161551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7gt\" (UniqueName: \"kubernetes.io/projected/4f4fc37f-91c3-47c2-b389-2d09a2a76029-kube-api-access-ts7gt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262395 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262484 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262496 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.262559 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.262551 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.264433 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.264405 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f4fc37f-91c3-47c2-b389-2d09a2a76029-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.264779 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.264759 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f4fc37f-91c3-47c2-b389-2d09a2a76029-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.270327 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.270305 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7gt\" (UniqueName: \"kubernetes.io/projected/4f4fc37f-91c3-47c2-b389-2d09a2a76029-kube-api-access-ts7gt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h\" (UID: \"4f4fc37f-91c3-47c2-b389-2d09a2a76029\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.330303 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.330274 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:29.463046 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.463021 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h"] Jul 16 04:49:29.464484 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:49:29.464446 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4fc37f_91c3_47c2_b389_2d09a2a76029.slice/crio-2260f31ca9d61525bc05ed0b1f53a72004cbd4eb463da5fe948d08157d648302 WatchSource:0}: Error finding container 2260f31ca9d61525bc05ed0b1f53a72004cbd4eb463da5fe948d08157d648302: Status 404 returned error can't find the container with id 2260f31ca9d61525bc05ed0b1f53a72004cbd4eb463da5fe948d08157d648302 Jul 16 04:49:29.465691 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.465673 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 04:49:29.556488 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:29.556405 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" event={"ID":"4f4fc37f-91c3-47c2-b389-2d09a2a76029","Type":"ContainerStarted","Data":"2260f31ca9d61525bc05ed0b1f53a72004cbd4eb463da5fe948d08157d648302"} Jul 16 04:49:30.562731 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:30.562696 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" event={"ID":"4f4fc37f-91c3-47c2-b389-2d09a2a76029","Type":"ContainerStarted","Data":"298f16b14b36bb3c4af21a653c00f6030dd31378363f5c81bf3888bccd39e900"} Jul 16 04:49:30.581682 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:30.581629 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" podStartSLOduration=2.39864365 podStartE2EDuration="2.581613707s" totalImagesPullingTime="182.970057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:49:30.579322576 +0000 UTC m=+634.305692877" watchObservedRunningTime="2026-07-16 04:49:30.581613707 +0000 UTC m=+634.307984005" Jul 16 04:49:39.331117 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:39.331077 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:39.331546 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:39.331160 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:39.343981 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:39.343955 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:39.620506 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:39.620475 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h" Jul 16 04:49:52.905824 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.905788 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj"] Jul 16 04:49:52.909832 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.909809 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.912623 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.912595 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 16 04:49:52.919001 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.918976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj"] Jul 16 04:49:52.971186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971120 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.971186 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.971405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtq2\" (UniqueName: \"kubernetes.io/projected/b3931727-11e8-4529-9b19-0b90f25508d9-kube-api-access-ngtq2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.971405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971318 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.971405 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3931727-11e8-4529-9b19-0b90f25508d9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:52.971561 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:52.971426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.072635 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3931727-11e8-4529-9b19-0b90f25508d9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.072849 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072669 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.072849 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.072849 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.073019 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtq2\" (UniqueName: \"kubernetes.io/projected/b3931727-11e8-4529-9b19-0b90f25508d9-kube-api-access-ngtq2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.073019 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.072931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.073291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.073243 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.073291 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.073278 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.073468 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.073346 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.075239 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.075218 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3931727-11e8-4529-9b19-0b90f25508d9-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.075411 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.075394 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3931727-11e8-4529-9b19-0b90f25508d9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.081513 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.081493 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtq2\" (UniqueName: \"kubernetes.io/projected/b3931727-11e8-4529-9b19-0b90f25508d9-kube-api-access-ngtq2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj\" (UID: \"b3931727-11e8-4529-9b19-0b90f25508d9\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.222883 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.222785 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:49:53.359822 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.359793 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj"] Jul 16 04:49:53.362508 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:49:53.362477 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3931727_11e8_4529_9b19_0b90f25508d9.slice/crio-c356a4b61025779dc877ca7030531e32fe95fcf6a041299845d0c0cff3ebe6ad WatchSource:0}: Error finding container c356a4b61025779dc877ca7030531e32fe95fcf6a041299845d0c0cff3ebe6ad: Status 404 returned error can't find the container with id c356a4b61025779dc877ca7030531e32fe95fcf6a041299845d0c0cff3ebe6ad Jul 16 04:49:53.685165 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.685105 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" event={"ID":"b3931727-11e8-4529-9b19-0b90f25508d9","Type":"ContainerStarted","Data":"28873ee19d7e8b78aa41be2a18324b37a8977bb534857ea5b0f13945421f019d"} Jul 16 04:49:53.685362 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.685186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" event={"ID":"b3931727-11e8-4529-9b19-0b90f25508d9","Type":"ContainerStarted","Data":"c356a4b61025779dc877ca7030531e32fe95fcf6a041299845d0c0cff3ebe6ad"} Jul 16 04:49:53.705303 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:49:53.705242 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" podStartSLOduration=1.494878167 podStartE2EDuration="1.70522538s" totalImagesPullingTime="210.347213ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:49:53.703208088 +0000 UTC m=+657.429578389" watchObservedRunningTime="2026-07-16 04:49:53.70522538 +0000 UTC m=+657.431595679" Jul 16 04:50:03.223415 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:50:03.223377 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:50:03.223985 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:50:03.223538 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:50:03.236815 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:50:03.236787 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:50:03.759100 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:50:03.759069 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj" Jul 16 04:52:15.627772 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.627737 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:52:15.628411 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.627981 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" podUID="e3484f60-4294-4821-ac65-b29adb7a2ebc" containerName="manager" containerID="cri-o://ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888" gracePeriod=30 Jul 16 04:52:15.871890 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.871866 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:52:15.957047 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.956966 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhkvd\" (UniqueName: \"kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd\") pod \"e3484f60-4294-4821-ac65-b29adb7a2ebc\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " Jul 16 04:52:15.957213 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.957045 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert\") pod \"e3484f60-4294-4821-ac65-b29adb7a2ebc\" (UID: \"e3484f60-4294-4821-ac65-b29adb7a2ebc\") " Jul 16 04:52:15.959190 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.959160 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e3484f60-4294-4821-ac65-b29adb7a2ebc" (UID: "e3484f60-4294-4821-ac65-b29adb7a2ebc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 04:52:15.959308 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:15.959213 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd" (OuterVolumeSpecName: "kube-api-access-mhkvd") pod "e3484f60-4294-4821-ac65-b29adb7a2ebc" (UID: "e3484f60-4294-4821-ac65-b29adb7a2ebc"). InnerVolumeSpecName "kube-api-access-mhkvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:52:16.057980 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.057944 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhkvd\" (UniqueName: \"kubernetes.io/projected/e3484f60-4294-4821-ac65-b29adb7a2ebc-kube-api-access-mhkvd\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:52:16.057980 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.057975 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3484f60-4294-4821-ac65-b29adb7a2ebc-webhook-cert\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:52:16.451454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.451417 2573 generic.go:356] "Generic (PLEG): container finished" podID="e3484f60-4294-4821-ac65-b29adb7a2ebc" containerID="ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888" exitCode=0 Jul 16 04:52:16.451454 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.451455 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" Jul 16 04:52:16.451655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.451502 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" event={"ID":"e3484f60-4294-4821-ac65-b29adb7a2ebc","Type":"ContainerDied","Data":"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888"} Jul 16 04:52:16.451655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.451558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b6d84dc7c-j257m" event={"ID":"e3484f60-4294-4821-ac65-b29adb7a2ebc","Type":"ContainerDied","Data":"9f6e73f82a70efadcf9f04c7348195df64e894f5b4704d0012e2635f66dff4e4"} Jul 16 04:52:16.451655 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.451581 2573 scope.go:117] "RemoveContainer" containerID="ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888" Jul 16 04:52:16.461683 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.461647 2573 scope.go:117] "RemoveContainer" containerID="ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888" Jul 16 04:52:16.461958 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:52:16.461934 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888\": container with ID starting with ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888 not found: ID does not exist" containerID="ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888" Jul 16 04:52:16.462062 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.461962 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888"} err="failed to get container status \"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888\": rpc error: code = NotFound desc = could not find container \"ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888\": container with ID starting with ee81d272400b2d9d13e61ec4c6ecf53daef5b070746d33c41addd1438c946888 not found: ID does not exist" Jul 16 04:52:16.473219 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.473196 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:52:16.475832 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:16.475808 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b6d84dc7c-j257m"] Jul 16 04:52:17.513278 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:52:17.513242 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3484f60-4294-4821-ac65-b29adb7a2ebc" path="/var/lib/kubelet/pods/e3484f60-4294-4821-ac65-b29adb7a2ebc/volumes" Jul 16 04:53:56.799531 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:53:56.799502 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:53:56.803600 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:53:56.803575 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:57:08.347448 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:08.347369 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:57:08.347957 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:08.347556 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" podUID="50481031-d950-4868-bea4-389e987a1440" containerName="manager" containerID="cri-o://2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c" gracePeriod=10 Jul 16 04:57:12.301054 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.301028 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:57:12.444342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.444239 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k42g2\" (UniqueName: \"kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2\") pod \"50481031-d950-4868-bea4-389e987a1440\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " Jul 16 04:57:12.444342 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.444291 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume\") pod \"50481031-d950-4868-bea4-389e987a1440\" (UID: \"50481031-d950-4868-bea4-389e987a1440\") " Jul 16 04:57:12.444684 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.444662 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50481031-d950-4868-bea4-389e987a1440" (UID: "50481031-d950-4868-bea4-389e987a1440"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 04:57:12.446404 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.446381 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2" (OuterVolumeSpecName: "kube-api-access-k42g2") pod "50481031-d950-4868-bea4-389e987a1440" (UID: "50481031-d950-4868-bea4-389e987a1440"). InnerVolumeSpecName "kube-api-access-k42g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 04:57:12.545297 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.545261 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k42g2\" (UniqueName: \"kubernetes.io/projected/50481031-d950-4868-bea4-389e987a1440-kube-api-access-k42g2\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:57:12.545297 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:12.545291 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50481031-d950-4868-bea4-389e987a1440-extensions-socket-volume\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 04:57:13.037230 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.037194 2573 generic.go:356] "Generic (PLEG): container finished" podID="50481031-d950-4868-bea4-389e987a1440" containerID="2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c" exitCode=0 Jul 16 04:57:13.037408 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.037243 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" Jul 16 04:57:13.037408 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.037287 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" event={"ID":"50481031-d950-4868-bea4-389e987a1440","Type":"ContainerDied","Data":"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c"} Jul 16 04:57:13.037408 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.037349 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm" event={"ID":"50481031-d950-4868-bea4-389e987a1440","Type":"ContainerDied","Data":"8b32275f933dd2f8e5ae8e51df2bc02d6e3c50537decd5a565e44f69bdd91a24"} Jul 16 04:57:13.037408 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.037372 2573 scope.go:117] "RemoveContainer" containerID="2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c" Jul 16 04:57:13.047129 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.047106 2573 scope.go:117] "RemoveContainer" containerID="2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c" Jul 16 04:57:13.047388 ip-10-0-137-187 kubenswrapper[2573]: E0716 04:57:13.047366 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c\": container with ID starting with 2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c not found: ID does not exist" containerID="2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c" Jul 16 04:57:13.047436 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.047397 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c"} err="failed to get container status \"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c\": rpc error: code = NotFound desc = could not find container \"2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c\": container with ID starting with 2fe42fa7d7dd991639d72ee0563d45957af3c6f798f08f489ae2527d8409075c not found: ID does not exist" Jul 16 04:57:13.060895 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.060869 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:57:13.062934 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.062915 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qt6pm"] Jul 16 04:57:13.513428 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:57:13.513397 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50481031-d950-4868-bea4-389e987a1440" path="/var/lib/kubelet/pods/50481031-d950-4868-bea4-389e987a1440/volumes" Jul 16 04:58:09.456173 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456123 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw"] Jul 16 04:58:09.456580 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456497 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3484f60-4294-4821-ac65-b29adb7a2ebc" containerName="manager" Jul 16 04:58:09.456580 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456507 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3484f60-4294-4821-ac65-b29adb7a2ebc" containerName="manager" Jul 16 04:58:09.456580 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456518 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50481031-d950-4868-bea4-389e987a1440" containerName="manager" Jul 16 04:58:09.456580 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456524 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="50481031-d950-4868-bea4-389e987a1440" containerName="manager" Jul 16 04:58:09.456776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456616 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3484f60-4294-4821-ac65-b29adb7a2ebc" containerName="manager" Jul 16 04:58:09.456776 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.456625 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="50481031-d950-4868-bea4-389e987a1440" containerName="manager" Jul 16 04:58:09.459990 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.459973 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.462955 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.462929 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 04:58:09.463087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.462954 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 04:58:09.463087 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.462975 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dgrpj\"" Jul 16 04:58:09.472116 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.472090 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw"] Jul 16 04:58:09.474210 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.474183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.474329 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.474224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlq7x\" (UniqueName: \"kubernetes.io/projected/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-kube-api-access-mlq7x\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.575265 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.575237 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.575456 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.575280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlq7x\" (UniqueName: \"kubernetes.io/projected/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-kube-api-access-mlq7x\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.575672 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.575653 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.584032 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.584001 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlq7x\" (UniqueName: \"kubernetes.io/projected/9b0bc6c5-5080-433f-93f5-8946b5cba5cd-kube-api-access-mlq7x\") pod \"kuadrant-operator-controller-manager-86878c769f-6mvgw\" (UID: \"9b0bc6c5-5080-433f-93f5-8946b5cba5cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.770682 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.770581 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:09.912340 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:09.912312 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw"] Jul 16 04:58:09.913743 ip-10-0-137-187 kubenswrapper[2573]: W0716 04:58:09.913703 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0bc6c5_5080_433f_93f5_8946b5cba5cd.slice/crio-4776b8086851a79d79087986fa1c2ab824fd651fe75bea8f52567e36ac72c22e WatchSource:0}: Error finding container 4776b8086851a79d79087986fa1c2ab824fd651fe75bea8f52567e36ac72c22e: Status 404 returned error can't find the container with id 4776b8086851a79d79087986fa1c2ab824fd651fe75bea8f52567e36ac72c22e Jul 16 04:58:10.343164 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:10.343104 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" event={"ID":"9b0bc6c5-5080-433f-93f5-8946b5cba5cd","Type":"ContainerStarted","Data":"23c03800c8e5740039f528469b1fd12f0fc52e0cc5f981eac2cdfae186ab5bb7"} Jul 16 04:58:10.343345 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:10.343230 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" event={"ID":"9b0bc6c5-5080-433f-93f5-8946b5cba5cd","Type":"ContainerStarted","Data":"4776b8086851a79d79087986fa1c2ab824fd651fe75bea8f52567e36ac72c22e"} Jul 16 04:58:10.343345 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:10.343263 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:10.369709 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:10.369653 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" podStartSLOduration=1.369638735 podStartE2EDuration="1.369638735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 04:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 04:58:10.366055802 +0000 UTC m=+1154.092426095" watchObservedRunningTime="2026-07-16 04:58:10.369638735 +0000 UTC m=+1154.096009033" Jul 16 04:58:21.352036 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:21.351956 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6mvgw" Jul 16 04:58:56.841619 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:56.841588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 04:58:56.842225 ip-10-0-137-187 kubenswrapper[2573]: I0716 04:58:56.842137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:03:56.886509 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:03:56.886480 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:03:56.887030 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:03:56.886867 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:08:56.923559 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:08:56.923523 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:08:56.925651 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:08:56.925628 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:09:16.604531 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.604494 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:09:16.607973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.607948 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.611450 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.611426 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-82a682\"/\"test-model-82a682-kserve-self-signed-certs\"" Jul 16 05:09:16.611450 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.611440 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-82a682\"/\"default-dockercfg-8qf5p\"" Jul 16 05:09:16.611722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.611527 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-82a682\"/\"kube-root-ca.crt\"" Jul 16 05:09:16.611722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.611554 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-82a682\"/\"openshift-service-ca.crt\"" Jul 16 05:09:16.619409 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.619383 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:09:16.787836 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.787794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.787836 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.787840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.788059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.787862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q62cq\" (UniqueName: \"kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.788059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.787954 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.788059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.787988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.788059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.788010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888615 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888581 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888615 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888765 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.888841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.888801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q62cq\" (UniqueName: \"kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.889084 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.889058 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.889182 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.889071 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.889182 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.889102 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.891132 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.891112 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.891324 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.891306 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.896777 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.896759 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q62cq\" (UniqueName: \"kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq\") pod \"test-model-82a682-kserve-5f9f5f8f47-4bmcw\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:16.917830 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:16.917798 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:17.061472 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:17.061446 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:09:17.062735 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:09:17.062707 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8121ff7a_c04c_4f91_ac54_c29f65897a26.slice/crio-9f8872036f6ce722593384baff0efef6d896618a7d4fb5042a72f43900893f63 WatchSource:0}: Error finding container 9f8872036f6ce722593384baff0efef6d896618a7d4fb5042a72f43900893f63: Status 404 returned error can't find the container with id 9f8872036f6ce722593384baff0efef6d896618a7d4fb5042a72f43900893f63 Jul 16 05:09:17.917604 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:17.917571 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" event={"ID":"8121ff7a-c04c-4f91-ac54-c29f65897a26","Type":"ContainerStarted","Data":"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b"} Jul 16 05:09:17.917604 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:17.917611 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" event={"ID":"8121ff7a-c04c-4f91-ac54-c29f65897a26","Type":"ContainerStarted","Data":"9f8872036f6ce722593384baff0efef6d896618a7d4fb5042a72f43900893f63"} Jul 16 05:09:17.934922 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:17.934849 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" podStartSLOduration=1.934832697 podStartE2EDuration="1.934832697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:09:17.93392555 +0000 UTC m=+1821.660295852" watchObservedRunningTime="2026-07-16 05:09:17.934832697 +0000 UTC m=+1821.661202996" Jul 16 05:09:26.918484 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:26.918450 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:26.919021 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:26.918603 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:26.931338 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:26.931311 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:26.979269 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:26.979239 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:09:41.583577 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.583542 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:09:41.586991 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.586969 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.589962 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.589938 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-05c504\"/\"kube-root-ca.crt\"" Jul 16 05:09:41.591169 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.591137 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-05c504\"/\"default-dockercfg-6bcsd\"" Jul 16 05:09:41.591275 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.591221 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-05c504\"/\"test-model-05c504-kserve-self-signed-certs\"" Jul 16 05:09:41.591275 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.591240 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-05c504\"/\"openshift-service-ca.crt\"" Jul 16 05:09:41.599937 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.599910 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:09:41.717383 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717347 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.717581 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.717581 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717424 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.717581 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717484 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.717581 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.717581 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.717537 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc6pr\" (UniqueName: \"kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.818770 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.818726 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.818989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.818780 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.818989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.818800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.818989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.818877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.819191 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.818999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.819191 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.819044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc6pr\" (UniqueName: \"kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.819326 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.819306 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.819393 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.819344 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.819453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.819405 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.821109 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.821090 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.821605 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.821585 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.827213 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.827187 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc6pr\" (UniqueName: \"kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr\") pod \"test-model-05c504-kserve-7cdb687747-5w67x\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:41.896463 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:41.896430 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:42.047632 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:42.047607 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:09:42.048940 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:09:42.048906 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeee9b9cb_054e_450f_99c8_471c4bf9c29e.slice/crio-6eae22e410d8ef2acfdff8bec0b5f0a27b3d485b596898e67be0c66828d8b1ee WatchSource:0}: Error finding container 6eae22e410d8ef2acfdff8bec0b5f0a27b3d485b596898e67be0c66828d8b1ee: Status 404 returned error can't find the container with id 6eae22e410d8ef2acfdff8bec0b5f0a27b3d485b596898e67be0c66828d8b1ee Jul 16 05:09:43.057817 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:43.057775 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" event={"ID":"eee9b9cb-054e-450f-99c8-471c4bf9c29e","Type":"ContainerStarted","Data":"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca"} Jul 16 05:09:43.058245 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:43.057827 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" event={"ID":"eee9b9cb-054e-450f-99c8-471c4bf9c29e","Type":"ContainerStarted","Data":"6eae22e410d8ef2acfdff8bec0b5f0a27b3d485b596898e67be0c66828d8b1ee"} Jul 16 05:09:43.081945 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:43.081889 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" podStartSLOduration=2.081870093 podStartE2EDuration="2.081870093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:09:43.080456431 +0000 UTC m=+1846.806826732" watchObservedRunningTime="2026-07-16 05:09:43.081870093 +0000 UTC m=+1846.808240393" Jul 16 05:09:51.896744 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:51.896706 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:51.897167 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:51.896758 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:51.909308 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:51.909276 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:09:52.120272 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:09:52.120232 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:10:01.682426 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:01.682388 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:10:02.690451 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:02.690417 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:10:16.702900 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:16.702855 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" podUID="8121ff7a-c04c-4f91-ac54-c29f65897a26" containerName="main" containerID="cri-o://4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b" gracePeriod=45 Jul 16 05:10:16.959795 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:16.959729 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:10:17.048226 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048183 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q62cq\" (UniqueName: \"kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048226 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048222 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048317 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048359 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048390 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048419 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir\") pod \"8121ff7a-c04c-4f91-ac54-c29f65897a26\" (UID: \"8121ff7a-c04c-4f91-ac54-c29f65897a26\") " Jul 16 05:10:17.048681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048528 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache" (OuterVolumeSpecName: "model-cache") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:17.048681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048633 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home" (OuterVolumeSpecName: "home") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:17.048799 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048741 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:17.048840 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048804 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.048840 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.048820 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.050644 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.050616 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq" (OuterVolumeSpecName: "kube-api-access-q62cq") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "kube-api-access-q62cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:10:17.050644 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.050608 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm" (OuterVolumeSpecName: "dshm") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:17.050799 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.050639 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8121ff7a-c04c-4f91-ac54-c29f65897a26" (UID: "8121ff7a-c04c-4f91-ac54-c29f65897a26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:10:17.149722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.149680 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.149722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.149718 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8121ff7a-c04c-4f91-ac54-c29f65897a26-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.149722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.149728 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8121ff7a-c04c-4f91-ac54-c29f65897a26-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.149722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.149737 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q62cq\" (UniqueName: \"kubernetes.io/projected/8121ff7a-c04c-4f91-ac54-c29f65897a26-kube-api-access-q62cq\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:17.251079 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.250989 2573 generic.go:356] "Generic (PLEG): container finished" podID="8121ff7a-c04c-4f91-ac54-c29f65897a26" containerID="4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b" exitCode=0 Jul 16 05:10:17.251079 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.251030 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" Jul 16 05:10:17.251079 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.251060 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" event={"ID":"8121ff7a-c04c-4f91-ac54-c29f65897a26","Type":"ContainerDied","Data":"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b"} Jul 16 05:10:17.251327 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.251092 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw" event={"ID":"8121ff7a-c04c-4f91-ac54-c29f65897a26","Type":"ContainerDied","Data":"9f8872036f6ce722593384baff0efef6d896618a7d4fb5042a72f43900893f63"} Jul 16 05:10:17.251327 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.251108 2573 scope.go:117] "RemoveContainer" containerID="4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b" Jul 16 05:10:17.260702 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.260683 2573 scope.go:117] "RemoveContainer" containerID="4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b" Jul 16 05:10:17.260951 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:10:17.260936 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b\": container with ID starting with 4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b not found: ID does not exist" containerID="4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b" Jul 16 05:10:17.260990 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.260959 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b"} err="failed to get container status \"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b\": rpc error: code = NotFound desc = could not find container \"4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b\": container with ID starting with 4a9e2c215dde90ab1ce984b75e73d8f90f369956f505b8c17b658057b44f9e4b not found: ID does not exist" Jul 16 05:10:17.273898 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.273871 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:10:17.278107 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.278085 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-82a682/test-model-82a682-kserve-5f9f5f8f47-4bmcw"] Jul 16 05:10:17.514610 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.514522 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8121ff7a-c04c-4f91-ac54-c29f65897a26" path="/var/lib/kubelet/pods/8121ff7a-c04c-4f91-ac54-c29f65897a26/volumes" Jul 16 05:10:17.710740 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.710690 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" podUID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" containerName="main" containerID="cri-o://8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca" gracePeriod=45 Jul 16 05:10:17.966304 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:17.966273 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:10:18.059803 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059765 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.059803 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059803 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.060027 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059870 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.060027 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059910 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.060027 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059946 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc6pr\" (UniqueName: \"kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.060027 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.059977 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home\") pod \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\" (UID: \"eee9b9cb-054e-450f-99c8-471c4bf9c29e\") " Jul 16 05:10:18.060262 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.060094 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:18.060320 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.060242 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache" (OuterVolumeSpecName: "model-cache") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:18.060320 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.060302 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.060388 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.060335 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home" (OuterVolumeSpecName: "home") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:18.062170 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.062122 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm" (OuterVolumeSpecName: "dshm") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:10:18.062279 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.062171 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:10:18.062320 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.062272 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr" (OuterVolumeSpecName: "kube-api-access-xc6pr") pod "eee9b9cb-054e-450f-99c8-471c4bf9c29e" (UID: "eee9b9cb-054e-450f-99c8-471c4bf9c29e"). InnerVolumeSpecName "kube-api-access-xc6pr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:10:18.161086 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.161043 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.161086 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.161074 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eee9b9cb-054e-450f-99c8-471c4bf9c29e-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.161086 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.161085 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc6pr\" (UniqueName: \"kubernetes.io/projected/eee9b9cb-054e-450f-99c8-471c4bf9c29e-kube-api-access-xc6pr\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.161086 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.161094 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.161505 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.161104 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eee9b9cb-054e-450f-99c8-471c4bf9c29e-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:10:18.257395 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.257359 2573 generic.go:356] "Generic (PLEG): container finished" podID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" containerID="8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca" exitCode=0 Jul 16 05:10:18.257577 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.257405 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" Jul 16 05:10:18.257577 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.257444 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" event={"ID":"eee9b9cb-054e-450f-99c8-471c4bf9c29e","Type":"ContainerDied","Data":"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca"} Jul 16 05:10:18.257577 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.257490 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x" event={"ID":"eee9b9cb-054e-450f-99c8-471c4bf9c29e","Type":"ContainerDied","Data":"6eae22e410d8ef2acfdff8bec0b5f0a27b3d485b596898e67be0c66828d8b1ee"} Jul 16 05:10:18.257577 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.257507 2573 scope.go:117] "RemoveContainer" containerID="8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca" Jul 16 05:10:18.266718 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.266702 2573 scope.go:117] "RemoveContainer" containerID="8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca" Jul 16 05:10:18.266946 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:10:18.266930 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca\": container with ID starting with 8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca not found: ID does not exist" containerID="8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca" Jul 16 05:10:18.266998 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.266954 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca"} err="failed to get container status \"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca\": rpc error: code = NotFound desc = could not find container \"8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca\": container with ID starting with 8929331ab3e692aae3fb41415ec0b9b623efbfc00822e6b66e072a0b43855eca not found: ID does not exist" Jul 16 05:10:18.278101 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.278076 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:10:18.285352 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:18.285330 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-05c504/test-model-05c504-kserve-7cdb687747-5w67x"] Jul 16 05:10:19.514257 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:10:19.514182 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" path="/var/lib/kubelet/pods/eee9b9cb-054e-450f-99c8-471c4bf9c29e/volumes" Jul 16 05:12:27.617093 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617061 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:12:27.617599 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617522 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" containerName="main" Jul 16 05:12:27.617599 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617549 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" containerName="main" Jul 16 05:12:27.617599 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617581 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8121ff7a-c04c-4f91-ac54-c29f65897a26" containerName="main" Jul 16 05:12:27.617599 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617590 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8121ff7a-c04c-4f91-ac54-c29f65897a26" containerName="main" Jul 16 05:12:27.617780 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617672 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8121ff7a-c04c-4f91-ac54-c29f65897a26" containerName="main" Jul 16 05:12:27.617780 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.617681 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee9b9cb-054e-450f-99c8-471c4bf9c29e" containerName="main" Jul 16 05:12:27.625375 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.625348 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.629051 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.629032 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"auth-test-model-a01a82-kserve-self-signed-certs\"" Jul 16 05:12:27.629217 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.629075 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"openshift-service-ca.crt\"" Jul 16 05:12:27.629217 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.629094 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"default-dockercfg-xm6kx\"" Jul 16 05:12:27.629217 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.629029 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"kube-root-ca.crt\"" Jul 16 05:12:27.635131 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.635105 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:12:27.724989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.724950 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.724989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.724995 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.725252 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.725020 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.725252 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.725043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.725252 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.725127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2khs\" (UniqueName: \"kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.725252 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.725206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825661 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825621 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2khs\" (UniqueName: \"kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825661 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825664 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825938 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825938 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825938 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.825938 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.825774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.826206 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.826136 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.826332 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.826176 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.826332 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.826302 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.828098 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.828078 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.828370 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.828353 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.834157 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.834114 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2khs\" (UniqueName: \"kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs\") pod \"auth-test-model-a01a82-kserve-d8948867-4rhdw\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:27.939017 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:27.938927 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:28.077624 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:28.077596 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:12:28.080221 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:12:28.080188 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded9ec4f8_93e5_4f77_9778_2c556560826c.slice/crio-2fe0969a074a52bc1169dce88346485a57695c8ce121db10ff84131e107e851c WatchSource:0}: Error finding container 2fe0969a074a52bc1169dce88346485a57695c8ce121db10ff84131e107e851c: Status 404 returned error can't find the container with id 2fe0969a074a52bc1169dce88346485a57695c8ce121db10ff84131e107e851c Jul 16 05:12:28.977131 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:28.977089 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" event={"ID":"ed9ec4f8-93e5-4f77-9778-2c556560826c","Type":"ContainerStarted","Data":"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad"} Jul 16 05:12:28.977557 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:28.977159 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" event={"ID":"ed9ec4f8-93e5-4f77-9778-2c556560826c","Type":"ContainerStarted","Data":"2fe0969a074a52bc1169dce88346485a57695c8ce121db10ff84131e107e851c"} Jul 16 05:12:28.995823 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:28.995773 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" podStartSLOduration=1.9957578969999998 podStartE2EDuration="1.995757897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:12:28.993217462 +0000 UTC m=+2012.719587761" watchObservedRunningTime="2026-07-16 05:12:28.995757897 +0000 UTC m=+2012.722128195" Jul 16 05:12:37.939580 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:37.939540 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:37.940084 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:37.939687 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:37.955931 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:37.955902 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:38.036383 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:38.036356 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:12:48.804118 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.804085 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:12:48.807957 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.807936 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.810407 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.810379 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"kube-root-ca.crt\"" Jul 16 05:12:48.811325 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.811306 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"default-dockercfg-nztgs\"" Jul 16 05:12:48.811396 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.811344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"auth-test-model-12efc5-kserve-self-signed-certs\"" Jul 16 05:12:48.811396 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.811371 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"openshift-service-ca.crt\"" Jul 16 05:12:48.817078 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.817054 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:12:48.932091 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.932091 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.932371 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.932371 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932328 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.932478 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:48.932478 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:48.932396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9pp\" (UniqueName: \"kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033358 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033318 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033584 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033374 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033584 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033584 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033584 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033584 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9pp\" (UniqueName: \"kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033870 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033826 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033870 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033858 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.033947 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.033899 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.035778 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.035758 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.036119 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.036103 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.041237 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.041213 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9pp\" (UniqueName: \"kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp\") pod \"auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.119262 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.119233 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:49.259029 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:49.259000 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:12:49.261013 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:12:49.260977 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1807c17_1443_4757_a1f3_1b75ecb9658d.slice/crio-fcd10b7da17fdbb6f6f9926af9962a0bd2dd416ac5ef60a8c99975b89a135abf WatchSource:0}: Error finding container fcd10b7da17fdbb6f6f9926af9962a0bd2dd416ac5ef60a8c99975b89a135abf: Status 404 returned error can't find the container with id fcd10b7da17fdbb6f6f9926af9962a0bd2dd416ac5ef60a8c99975b89a135abf Jul 16 05:12:50.090127 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:50.090090 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" event={"ID":"d1807c17-1443-4757-a1f3-1b75ecb9658d","Type":"ContainerStarted","Data":"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68"} Jul 16 05:12:50.090127 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:50.090132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" event={"ID":"d1807c17-1443-4757-a1f3-1b75ecb9658d","Type":"ContainerStarted","Data":"fcd10b7da17fdbb6f6f9926af9962a0bd2dd416ac5ef60a8c99975b89a135abf"} Jul 16 05:12:50.110290 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:50.110240 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" podStartSLOduration=2.110224456 podStartE2EDuration="2.110224456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:12:50.107656794 +0000 UTC m=+2033.834027095" watchObservedRunningTime="2026-07-16 05:12:50.110224456 +0000 UTC m=+2033.836594821" Jul 16 05:12:59.120191 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:59.120126 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:59.120191 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:59.120199 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:59.132597 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:59.132570 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:12:59.158762 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:12:59.158724 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:13:56.962988 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:13:56.962907 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:13:56.967526 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:13:56.967500 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:14:40.020129 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:40.020088 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:14:41.090912 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:41.090880 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:14:55.040472 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.040412 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" podUID="ed9ec4f8-93e5-4f77-9778-2c556560826c" containerName="main" containerID="cri-o://720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad" gracePeriod=45 Jul 16 05:14:55.301366 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.301302 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:14:55.393555 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393506 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.393555 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393562 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2khs\" (UniqueName: \"kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.393809 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393587 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.393809 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393617 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.393809 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393647 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.393809 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393699 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs\") pod \"ed9ec4f8-93e5-4f77-9778-2c556560826c\" (UID: \"ed9ec4f8-93e5-4f77-9778-2c556560826c\") " Jul 16 05:14:55.394020 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393814 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home" (OuterVolumeSpecName: "home") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:55.394020 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393913 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:55.394020 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.393975 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:55.394196 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.394092 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.394196 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.394113 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.394196 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.394128 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.395917 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.395891 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm" (OuterVolumeSpecName: "dshm") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:55.395989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.395917 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:14:55.396033 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.396014 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs" (OuterVolumeSpecName: "kube-api-access-v2khs") pod "ed9ec4f8-93e5-4f77-9778-2c556560826c" (UID: "ed9ec4f8-93e5-4f77-9778-2c556560826c"). InnerVolumeSpecName "kube-api-access-v2khs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:14:55.495192 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.495125 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2khs\" (UniqueName: \"kubernetes.io/projected/ed9ec4f8-93e5-4f77-9778-2c556560826c-kube-api-access-v2khs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.495192 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.495186 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed9ec4f8-93e5-4f77-9778-2c556560826c-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.495192 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.495202 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed9ec4f8-93e5-4f77-9778-2c556560826c-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:55.783499 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.783460 2573 generic.go:356] "Generic (PLEG): container finished" podID="ed9ec4f8-93e5-4f77-9778-2c556560826c" containerID="720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad" exitCode=0 Jul 16 05:14:55.783681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.783526 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" Jul 16 05:14:55.783681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.783544 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" event={"ID":"ed9ec4f8-93e5-4f77-9778-2c556560826c","Type":"ContainerDied","Data":"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad"} Jul 16 05:14:55.783681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.783588 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw" event={"ID":"ed9ec4f8-93e5-4f77-9778-2c556560826c","Type":"ContainerDied","Data":"2fe0969a074a52bc1169dce88346485a57695c8ce121db10ff84131e107e851c"} Jul 16 05:14:55.783681 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.783608 2573 scope.go:117] "RemoveContainer" containerID="720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad" Jul 16 05:14:55.794312 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.794287 2573 scope.go:117] "RemoveContainer" containerID="720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad" Jul 16 05:14:55.794597 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:14:55.794577 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad\": container with ID starting with 720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad not found: ID does not exist" containerID="720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad" Jul 16 05:14:55.794676 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.794611 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad"} err="failed to get container status \"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad\": rpc error: code = NotFound desc = could not find container \"720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad\": container with ID starting with 720477004b60fb26848bc56894fdc58ec9f492eda72db0073dd814fc31974dad not found: ID does not exist" Jul 16 05:14:55.803801 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.803776 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:14:55.807292 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:55.807266 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/auth-test-model-a01a82-kserve-d8948867-4rhdw"] Jul 16 05:14:56.111353 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.111289 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" podUID="d1807c17-1443-4757-a1f3-1b75ecb9658d" containerName="main" containerID="cri-o://a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68" gracePeriod=45 Jul 16 05:14:56.355887 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.355856 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:14:56.403756 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403678 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.403756 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403749 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.403966 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403780 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.403966 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403803 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.403966 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403875 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.403966 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403904 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9pp\" (UniqueName: \"kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp\") pod \"d1807c17-1443-4757-a1f3-1b75ecb9658d\" (UID: \"d1807c17-1443-4757-a1f3-1b75ecb9658d\") " Jul 16 05:14:56.404140 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.403966 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:56.404140 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.404053 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:56.404256 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.404136 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home" (OuterVolumeSpecName: "home") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:56.404315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.404277 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.404315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.404293 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.404315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.404304 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.406056 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.406027 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm" (OuterVolumeSpecName: "dshm") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:14:56.406167 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.406063 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp" (OuterVolumeSpecName: "kube-api-access-gx9pp") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "kube-api-access-gx9pp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:14:56.406167 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.406092 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1807c17-1443-4757-a1f3-1b75ecb9658d" (UID: "d1807c17-1443-4757-a1f3-1b75ecb9658d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:14:56.505365 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.505331 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1807c17-1443-4757-a1f3-1b75ecb9658d-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.505365 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.505358 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1807c17-1443-4757-a1f3-1b75ecb9658d-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.505365 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.505368 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9pp\" (UniqueName: \"kubernetes.io/projected/d1807c17-1443-4757-a1f3-1b75ecb9658d-kube-api-access-gx9pp\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:14:56.790315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.790221 2573 generic.go:356] "Generic (PLEG): container finished" podID="d1807c17-1443-4757-a1f3-1b75ecb9658d" containerID="a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68" exitCode=0 Jul 16 05:14:56.790315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.790260 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" Jul 16 05:14:56.790315 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.790301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" event={"ID":"d1807c17-1443-4757-a1f3-1b75ecb9658d","Type":"ContainerDied","Data":"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68"} Jul 16 05:14:56.790578 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.790351 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm" event={"ID":"d1807c17-1443-4757-a1f3-1b75ecb9658d","Type":"ContainerDied","Data":"fcd10b7da17fdbb6f6f9926af9962a0bd2dd416ac5ef60a8c99975b89a135abf"} Jul 16 05:14:56.790578 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.790372 2573 scope.go:117] "RemoveContainer" containerID="a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68" Jul 16 05:14:56.800275 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.800229 2573 scope.go:117] "RemoveContainer" containerID="a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68" Jul 16 05:14:56.800502 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:14:56.800480 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68\": container with ID starting with a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68 not found: ID does not exist" containerID="a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68" Jul 16 05:14:56.800548 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.800511 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68"} err="failed to get container status \"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68\": rpc error: code = NotFound desc = could not find container \"a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68\": container with ID starting with a656ca463ab8556b64d6b2ecca138c1b9a6b25d826165b1f6b9bfe2e50eb0a68 not found: ID does not exist" Jul 16 05:14:56.813724 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.813697 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:14:56.817497 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:56.817475 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/auth-test-model-12efc5-kserve-6ddb7b4c59-x4tzm"] Jul 16 05:14:57.513823 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:57.513790 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1807c17-1443-4757-a1f3-1b75ecb9658d" path="/var/lib/kubelet/pods/d1807c17-1443-4757-a1f3-1b75ecb9658d/volumes" Jul 16 05:14:57.514297 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:14:57.514278 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed9ec4f8-93e5-4f77-9778-2c556560826c" path="/var/lib/kubelet/pods/ed9ec4f8-93e5-4f77-9778-2c556560826c/volumes" Jul 16 05:15:00.282011 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.281971 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:15:00.282630 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282606 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1807c17-1443-4757-a1f3-1b75ecb9658d" containerName="main" Jul 16 05:15:00.282783 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282671 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1807c17-1443-4757-a1f3-1b75ecb9658d" containerName="main" Jul 16 05:15:00.282783 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282697 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed9ec4f8-93e5-4f77-9778-2c556560826c" containerName="main" Jul 16 05:15:00.282783 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282707 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed9ec4f8-93e5-4f77-9778-2c556560826c" containerName="main" Jul 16 05:15:00.282949 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282831 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1807c17-1443-4757-a1f3-1b75ecb9658d" containerName="main" Jul 16 05:15:00.282949 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.282849 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed9ec4f8-93e5-4f77-9778-2c556560826c" containerName="main" Jul 16 05:15:00.287617 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.287593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.291135 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.291110 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"sub-test-model-a01a82-kserve-self-signed-certs\"" Jul 16 05:15:00.291928 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.291909 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"default-dockercfg-xm6kx\"" Jul 16 05:15:00.302841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.302810 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:15:00.306298 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.306275 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"kube-root-ca.crt\"" Jul 16 05:15:00.306506 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.306483 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a01a82\"/\"openshift-service-ca.crt\"" Jul 16 05:15:00.341104 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.341104 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.341379 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.341379 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341196 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.341379 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.341379 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.341236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwf52\" (UniqueName: \"kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442231 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442231 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442235 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442520 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442348 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442520 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwf52\" (UniqueName: \"kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442520 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442688 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442542 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442688 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442676 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442772 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442694 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.442807 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.442799 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.444552 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.444530 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.444882 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.444866 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.453520 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.453496 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwf52\" (UniqueName: \"kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52\") pod \"sub-test-model-a01a82-kserve-bf54d5dfc-fxl96\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.597878 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.597772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:00.738279 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.738250 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:15:00.739780 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:15:00.739750 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod262206b1_f97a_4e6c_aa0f_6bb63f409efa.slice/crio-f3ecb56206b327cf8e0da42fc4eed21cc91758aa79abc7c1c7969daf927ffc61 WatchSource:0}: Error finding container f3ecb56206b327cf8e0da42fc4eed21cc91758aa79abc7c1c7969daf927ffc61: Status 404 returned error can't find the container with id f3ecb56206b327cf8e0da42fc4eed21cc91758aa79abc7c1c7969daf927ffc61 Jul 16 05:15:00.816317 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.816277 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" event={"ID":"262206b1-f97a-4e6c-aa0f-6bb63f409efa","Type":"ContainerStarted","Data":"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d"} Jul 16 05:15:00.816530 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.816326 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" event={"ID":"262206b1-f97a-4e6c-aa0f-6bb63f409efa","Type":"ContainerStarted","Data":"f3ecb56206b327cf8e0da42fc4eed21cc91758aa79abc7c1c7969daf927ffc61"} Jul 16 05:15:00.834088 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:00.834033 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" podStartSLOduration=0.833998492 podStartE2EDuration="833.998492ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:15:00.832679018 +0000 UTC m=+2164.559049319" watchObservedRunningTime="2026-07-16 05:15:00.833998492 +0000 UTC m=+2164.560368790" Jul 16 05:15:10.598169 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:10.598110 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:10.598582 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:10.598182 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:10.610734 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:10.610702 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:10.886789 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:10.886756 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:15:22.606320 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.606283 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:15:22.609453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.609438 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.612897 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.612865 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"kube-root-ca.crt\"" Jul 16 05:15:22.613049 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.612897 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"default-dockercfg-nztgs\"" Jul 16 05:15:22.613049 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.612911 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"openshift-service-ca.crt\"" Jul 16 05:15:22.613049 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.612915 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"sub-test-model-12efc5-kserve-self-signed-certs\"" Jul 16 05:15:22.619562 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.619534 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:15:22.751196 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751126 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.751384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751205 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.751384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751237 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.751384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751255 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.751384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.751384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.751353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gcv8\" (UniqueName: \"kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.852772 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.852734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.852772 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.852779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853026 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.852813 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853026 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.852839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853026 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.852989 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853225 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.853049 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gcv8\" (UniqueName: \"kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853225 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.853205 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853351 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.853250 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.853351 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.853323 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.855200 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.855177 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.855596 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.855576 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.862069 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.862009 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gcv8\" (UniqueName: \"kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8\") pod \"sub-test-model-12efc5-kserve-6c85c996cf-jgf5l\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:22.920073 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:22.920040 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:23.065808 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:23.065772 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:15:23.067270 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:15:23.067241 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fe7f8c4_d60e_4329_b22d_c32d728789c0.slice/crio-e159738be411bb2d0016450950809f156497dbc516f0d257499bf15118452557 WatchSource:0}: Error finding container e159738be411bb2d0016450950809f156497dbc516f0d257499bf15118452557: Status 404 returned error can't find the container with id e159738be411bb2d0016450950809f156497dbc516f0d257499bf15118452557 Jul 16 05:15:23.949408 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:23.949371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" event={"ID":"0fe7f8c4-d60e-4329-b22d-c32d728789c0","Type":"ContainerStarted","Data":"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1"} Jul 16 05:15:23.949865 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:23.949416 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" event={"ID":"0fe7f8c4-d60e-4329-b22d-c32d728789c0","Type":"ContainerStarted","Data":"e159738be411bb2d0016450950809f156497dbc516f0d257499bf15118452557"} Jul 16 05:15:23.967199 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:23.967131 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" podStartSLOduration=1.9671164110000001 podStartE2EDuration="1.967116411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:15:23.966090349 +0000 UTC m=+2187.692460648" watchObservedRunningTime="2026-07-16 05:15:23.967116411 +0000 UTC m=+2187.693486720" Jul 16 05:15:32.921103 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:32.921067 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:32.921524 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:32.921127 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:32.935519 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:32.935495 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:15:33.012969 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:15:33.012935 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:16:06.380973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:06.380936 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:16:07.309006 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:07.308965 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:16:21.401790 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.401682 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" podUID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" containerName="main" containerID="cri-o://9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d" gracePeriod=45 Jul 16 05:16:21.654473 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.654413 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:16:21.682823 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.682788 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.682981 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.682851 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.682981 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.682895 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.682981 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.682934 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.682981 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.682975 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.683236 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683014 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwf52\" (UniqueName: \"kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52\") pod \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\" (UID: \"262206b1-f97a-4e6c-aa0f-6bb63f409efa\") " Jul 16 05:16:21.683236 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683210 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache" (OuterVolumeSpecName: "model-cache") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:21.683371 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683348 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home" (OuterVolumeSpecName: "home") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:21.683533 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683470 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:21.683533 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683500 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:21.683722 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.683536 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:21.684997 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.684972 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm" (OuterVolumeSpecName: "dshm") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:21.685288 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.685269 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52" (OuterVolumeSpecName: "kube-api-access-zwf52") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "kube-api-access-zwf52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:16:21.685402 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.685385 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "262206b1-f97a-4e6c-aa0f-6bb63f409efa" (UID: "262206b1-f97a-4e6c-aa0f-6bb63f409efa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:16:21.783971 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.783930 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwf52\" (UniqueName: \"kubernetes.io/projected/262206b1-f97a-4e6c-aa0f-6bb63f409efa-kube-api-access-zwf52\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:21.783971 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.783965 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:21.783971 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.783976 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:21.783971 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:21.783985 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/262206b1-f97a-4e6c-aa0f-6bb63f409efa-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.266935 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.266901 2573 generic.go:356] "Generic (PLEG): container finished" podID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" containerID="9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d" exitCode=0 Jul 16 05:16:22.267126 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.266946 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" event={"ID":"262206b1-f97a-4e6c-aa0f-6bb63f409efa","Type":"ContainerDied","Data":"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d"} Jul 16 05:16:22.267126 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.266952 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" Jul 16 05:16:22.267126 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.266977 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96" event={"ID":"262206b1-f97a-4e6c-aa0f-6bb63f409efa","Type":"ContainerDied","Data":"f3ecb56206b327cf8e0da42fc4eed21cc91758aa79abc7c1c7969daf927ffc61"} Jul 16 05:16:22.267126 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.266995 2573 scope.go:117] "RemoveContainer" containerID="9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d" Jul 16 05:16:22.276691 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.276671 2573 scope.go:117] "RemoveContainer" containerID="9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d" Jul 16 05:16:22.276968 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:16:22.276949 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d\": container with ID starting with 9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d not found: ID does not exist" containerID="9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d" Jul 16 05:16:22.277014 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.276977 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d"} err="failed to get container status \"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d\": rpc error: code = NotFound desc = could not find container \"9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d\": container with ID starting with 9dc7b2e67c667834a33d4286974eee77cf51f3b167ba18cbf4ae030bd6b1c81d not found: ID does not exist" Jul 16 05:16:22.290430 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.290391 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:16:22.293826 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.293803 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a01a82/sub-test-model-a01a82-kserve-bf54d5dfc-fxl96"] Jul 16 05:16:22.329374 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.329337 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" podUID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" containerName="main" containerID="cri-o://fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1" gracePeriod=45 Jul 16 05:16:22.584203 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.584177 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:16:22.689783 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689749 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gcv8\" (UniqueName: \"kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.689973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689801 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.689973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689832 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.689973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689847 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.689973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689896 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.689973 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.689915 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache\") pod \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\" (UID: \"0fe7f8c4-d60e-4329-b22d-c32d728789c0\") " Jul 16 05:16:22.690274 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.690224 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:22.690274 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.690250 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:22.690358 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.690220 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home" (OuterVolumeSpecName: "home") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:22.692032 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.692002 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm" (OuterVolumeSpecName: "dshm") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:16:22.692161 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.692080 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:16:22.692218 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.692175 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8" (OuterVolumeSpecName: "kube-api-access-8gcv8") pod "0fe7f8c4-d60e-4329-b22d-c32d728789c0" (UID: "0fe7f8c4-d60e-4329-b22d-c32d728789c0"). InnerVolumeSpecName "kube-api-access-8gcv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:16:22.790565 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790528 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.790565 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790562 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gcv8\" (UniqueName: \"kubernetes.io/projected/0fe7f8c4-d60e-4329-b22d-c32d728789c0-kube-api-access-8gcv8\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.790565 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790572 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.790818 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790581 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.790818 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790591 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:22.790818 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:22.790598 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fe7f8c4-d60e-4329-b22d-c32d728789c0-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:16:23.273533 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.273489 2573 generic.go:356] "Generic (PLEG): container finished" podID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" containerID="fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1" exitCode=0 Jul 16 05:16:23.273533 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.273537 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" Jul 16 05:16:23.273812 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.273572 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" event={"ID":"0fe7f8c4-d60e-4329-b22d-c32d728789c0","Type":"ContainerDied","Data":"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1"} Jul 16 05:16:23.273812 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.273613 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l" event={"ID":"0fe7f8c4-d60e-4329-b22d-c32d728789c0","Type":"ContainerDied","Data":"e159738be411bb2d0016450950809f156497dbc516f0d257499bf15118452557"} Jul 16 05:16:23.273812 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.273638 2573 scope.go:117] "RemoveContainer" containerID="fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1" Jul 16 05:16:23.285645 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.285614 2573 scope.go:117] "RemoveContainer" containerID="fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1" Jul 16 05:16:23.286010 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:16:23.285983 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1\": container with ID starting with fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1 not found: ID does not exist" containerID="fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1" Jul 16 05:16:23.286127 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.286017 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1"} err="failed to get container status \"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1\": rpc error: code = NotFound desc = could not find container \"fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1\": container with ID starting with fa1482cf0192eb5c40bcd66b412cc8b28db6d1c41fa4c5a5f4279819ceb55ba1 not found: ID does not exist" Jul 16 05:16:23.297323 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.297288 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:16:23.304318 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.304288 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/sub-test-model-12efc5-kserve-6c85c996cf-jgf5l"] Jul 16 05:16:23.514738 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.514701 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" path="/var/lib/kubelet/pods/0fe7f8c4-d60e-4329-b22d-c32d728789c0/volumes" Jul 16 05:16:23.515103 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:23.515089 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" path="/var/lib/kubelet/pods/262206b1-f97a-4e6c-aa0f-6bb63f409efa/volumes" Jul 16 05:16:43.998059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998027 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:16:43.998731 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998623 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" containerName="main" Jul 16 05:16:43.998731 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998644 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" containerName="main" Jul 16 05:16:43.998731 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998696 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" containerName="main" Jul 16 05:16:43.998731 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998704 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" containerName="main" Jul 16 05:16:43.998946 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998790 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="262206b1-f97a-4e6c-aa0f-6bb63f409efa" containerName="main" Jul 16 05:16:43.998946 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:43.998806 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fe7f8c4-d60e-4329-b22d-c32d728789c0" containerName="main" Jul 16 05:16:44.001774 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.001753 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.005329 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.005307 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"rate-test-model-12efc5-kserve-self-signed-certs\"" Jul 16 05:16:44.005466 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.005339 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"kube-root-ca.crt\"" Jul 16 05:16:44.005466 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.005386 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"default-dockercfg-nztgs\"" Jul 16 05:16:44.005466 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.005339 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-12efc5\"/\"openshift-service-ca.crt\"" Jul 16 05:16:44.020007 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.019802 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:16:44.080293 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.080501 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.080501 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080391 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.080501 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tltkt\" (UniqueName: \"kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.080658 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.080658 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.080553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181686 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181686 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181915 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181722 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181915 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181915 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.181915 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.181804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tltkt\" (UniqueName: \"kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.182194 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.182134 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.182261 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.182198 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.182261 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.182219 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.184118 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.184093 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.184399 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.184377 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.190070 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.190046 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tltkt\" (UniqueName: \"kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt\") pod \"rate-test-model-12efc5-kserve-595cc7688c-s89sg\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.322600 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.322499 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:44.456319 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:44.456290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:16:44.458159 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:16:44.458113 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39a5f6e3_9252_4d85_9dc8_973cd68f427a.slice/crio-081767f7326687a70244cf24ae8f31c95d1bc70a066f41a56c17260af6b9bfa3 WatchSource:0}: Error finding container 081767f7326687a70244cf24ae8f31c95d1bc70a066f41a56c17260af6b9bfa3: Status 404 returned error can't find the container with id 081767f7326687a70244cf24ae8f31c95d1bc70a066f41a56c17260af6b9bfa3 Jul 16 05:16:45.397753 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:45.397701 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" event={"ID":"39a5f6e3-9252-4d85-9dc8-973cd68f427a","Type":"ContainerStarted","Data":"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0"} Jul 16 05:16:45.398249 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:45.397762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" event={"ID":"39a5f6e3-9252-4d85-9dc8-973cd68f427a","Type":"ContainerStarted","Data":"081767f7326687a70244cf24ae8f31c95d1bc70a066f41a56c17260af6b9bfa3"} Jul 16 05:16:45.417784 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:45.417729 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" podStartSLOduration=2.417714206 podStartE2EDuration="2.417714206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:16:45.416203048 +0000 UTC m=+2269.142573346" watchObservedRunningTime="2026-07-16 05:16:45.417714206 +0000 UTC m=+2269.144084504" Jul 16 05:16:54.323328 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:54.323286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:54.323831 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:54.323354 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:54.336059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:54.336038 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:16:54.459086 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:16:54.459057 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:17:26.281869 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:26.281832 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:17:27.245181 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.245136 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087259666736"] Jul 16 05:17:27.248807 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.248790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:27.251535 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.251508 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xg77t\"" Jul 16 05:17:27.254529 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.254504 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087259666736"] Jul 16 05:17:27.288157 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.288123 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x6j5\" (UniqueName: \"kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5\") pod \"test-curl-7491-140087259666736\" (UID: \"a604b598-e537-4012-8182-14ab90899174\") " pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:27.389516 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.389466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x6j5\" (UniqueName: \"kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5\") pod \"test-curl-7491-140087259666736\" (UID: \"a604b598-e537-4012-8182-14ab90899174\") " pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:27.398394 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.398365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x6j5\" (UniqueName: \"kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5\") pod \"test-curl-7491-140087259666736\" (UID: \"a604b598-e537-4012-8182-14ab90899174\") " pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:27.559287 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.559200 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:27.693316 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.693290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087259666736"] Jul 16 05:17:27.694800 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:17:27.694766 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda604b598_e537_4012_8182_14ab90899174.slice/crio-e398386dfddcc03c3f3b6eaf27bb7d5afb894136abcc9259e7398c8e22ad2e71 WatchSource:0}: Error finding container e398386dfddcc03c3f3b6eaf27bb7d5afb894136abcc9259e7398c8e22ad2e71: Status 404 returned error can't find the container with id e398386dfddcc03c3f3b6eaf27bb7d5afb894136abcc9259e7398c8e22ad2e71 Jul 16 05:17:27.696034 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:27.696013 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 05:17:28.638631 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:28.638593 2573 generic.go:356] "Generic (PLEG): container finished" podID="a604b598-e537-4012-8182-14ab90899174" containerID="769faf597c1c5ec0d68b7da2467ea4110ff40cdc54a517befbc205745cbc72ab" exitCode=0 Jul 16 05:17:28.639013 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:28.638678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087259666736" event={"ID":"a604b598-e537-4012-8182-14ab90899174","Type":"ContainerDied","Data":"769faf597c1c5ec0d68b7da2467ea4110ff40cdc54a517befbc205745cbc72ab"} Jul 16 05:17:28.639013 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:28.638718 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087259666736" event={"ID":"a604b598-e537-4012-8182-14ab90899174","Type":"ContainerStarted","Data":"e398386dfddcc03c3f3b6eaf27bb7d5afb894136abcc9259e7398c8e22ad2e71"} Jul 16 05:17:29.778525 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.778501 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:29.814456 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.814429 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087259666736_a604b598-e537-4012-8182-14ab90899174/test-curl-7491-140087259666736/0.log" Jul 16 05:17:29.838489 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.838458 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087259666736"] Jul 16 05:17:29.842011 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.841988 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087259666736"] Jul 16 05:17:29.914305 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.914217 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x6j5\" (UniqueName: \"kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5\") pod \"a604b598-e537-4012-8182-14ab90899174\" (UID: \"a604b598-e537-4012-8182-14ab90899174\") " Jul 16 05:17:29.916354 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.916319 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5" (OuterVolumeSpecName: "kube-api-access-5x6j5") pod "a604b598-e537-4012-8182-14ab90899174" (UID: "a604b598-e537-4012-8182-14ab90899174"). InnerVolumeSpecName "kube-api-access-5x6j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:29.964159 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.964116 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087259666864"] Jul 16 05:17:29.964501 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.964487 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a604b598-e537-4012-8182-14ab90899174" containerName="test-curl-7491-140087259666736" Jul 16 05:17:29.964552 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.964503 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a604b598-e537-4012-8182-14ab90899174" containerName="test-curl-7491-140087259666736" Jul 16 05:17:29.964596 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.964586 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a604b598-e537-4012-8182-14ab90899174" containerName="test-curl-7491-140087259666736" Jul 16 05:17:29.967602 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.967586 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:29.973918 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:29.973894 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087259666864"] Jul 16 05:17:30.015733 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.015704 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfnkh\" (UniqueName: \"kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh\") pod \"test-curl-7491-140087259666864\" (UID: \"313d56a1-8990-49fd-8c06-88b993b58546\") " pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:30.015897 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.015839 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x6j5\" (UniqueName: \"kubernetes.io/projected/a604b598-e537-4012-8182-14ab90899174-kube-api-access-5x6j5\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:30.117049 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.117014 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfnkh\" (UniqueName: \"kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh\") pod \"test-curl-7491-140087259666864\" (UID: \"313d56a1-8990-49fd-8c06-88b993b58546\") " pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:30.128269 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.128233 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfnkh\" (UniqueName: \"kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh\") pod \"test-curl-7491-140087259666864\" (UID: \"313d56a1-8990-49fd-8c06-88b993b58546\") " pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:30.278720 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.278624 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:30.417212 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.415770 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087259666864"] Jul 16 05:17:30.652376 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.652334 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087259666864" event={"ID":"313d56a1-8990-49fd-8c06-88b993b58546","Type":"ContainerStarted","Data":"5475e0bf0d3ca52738553696e621ce5177fe8536f45e19ae33d4e8f94655e1d7"} Jul 16 05:17:30.653703 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.653675 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e398386dfddcc03c3f3b6eaf27bb7d5afb894136abcc9259e7398c8e22ad2e71" Jul 16 05:17:30.653858 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:30.653712 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:30.664362 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:17:30.664333 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-140087259666736\" is forbidden: User \"system:node:ip-10-0-137-187.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-187.ec2.internal' and this object" podUID="a604b598-e537-4012-8182-14ab90899174" pod="opendatahub/test-curl-7491-140087259666736" Jul 16 05:17:31.513544 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:31.513499 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a604b598-e537-4012-8182-14ab90899174" path="/var/lib/kubelet/pods/a604b598-e537-4012-8182-14ab90899174/volumes" Jul 16 05:17:31.659989 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:31.659948 2573 generic.go:356] "Generic (PLEG): container finished" podID="313d56a1-8990-49fd-8c06-88b993b58546" containerID="51ad86711c4724c0519f36a6113d9dad46b93c5d2c5ab84a54764ed7c6ae0a15" exitCode=0 Jul 16 05:17:31.660273 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:31.660035 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087259666864" event={"ID":"313d56a1-8990-49fd-8c06-88b993b58546","Type":"ContainerDied","Data":"51ad86711c4724c0519f36a6113d9dad46b93c5d2c5ab84a54764ed7c6ae0a15"} Jul 16 05:17:32.802421 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.802397 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:32.821519 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.821481 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087259666864_313d56a1-8990-49fd-8c06-88b993b58546/test-curl-7491-140087259666864/0.log" Jul 16 05:17:32.843903 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.843870 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087259666864"] Jul 16 05:17:32.847193 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.847165 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087259666864"] Jul 16 05:17:32.945186 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.945080 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfnkh\" (UniqueName: \"kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh\") pod \"313d56a1-8990-49fd-8c06-88b993b58546\" (UID: \"313d56a1-8990-49fd-8c06-88b993b58546\") " Jul 16 05:17:32.947195 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.947171 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh" (OuterVolumeSpecName: "kube-api-access-jfnkh") pod "313d56a1-8990-49fd-8c06-88b993b58546" (UID: "313d56a1-8990-49fd-8c06-88b993b58546"). InnerVolumeSpecName "kube-api-access-jfnkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:32.991475 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.991441 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:32.991854 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.991840 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="313d56a1-8990-49fd-8c06-88b993b58546" containerName="test-curl-7491-140087259666864" Jul 16 05:17:32.991904 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.991855 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="313d56a1-8990-49fd-8c06-88b993b58546" containerName="test-curl-7491-140087259666864" Jul 16 05:17:32.991946 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.991926 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="313d56a1-8990-49fd-8c06-88b993b58546" containerName="test-curl-7491-140087259666864" Jul 16 05:17:32.994892 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:32.994871 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:33.000510 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.000483 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:33.046437 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.046399 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfnkh\" (UniqueName: \"kubernetes.io/projected/313d56a1-8990-49fd-8c06-88b993b58546-kube-api-access-jfnkh\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:33.147319 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.147283 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4vvc\" (UniqueName: \"kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc\") pod \"test-curl-7491-140087261838912\" (UID: \"14ac5079-cf8a-4d36-bf7c-543f95343143\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:33.247831 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.247748 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4vvc\" (UniqueName: \"kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc\") pod \"test-curl-7491-140087261838912\" (UID: \"14ac5079-cf8a-4d36-bf7c-543f95343143\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:33.261945 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.261912 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4vvc\" (UniqueName: \"kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc\") pod \"test-curl-7491-140087261838912\" (UID: \"14ac5079-cf8a-4d36-bf7c-543f95343143\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:33.305624 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.305588 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:33.438041 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.438016 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:33.439546 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:17:33.439512 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14ac5079_cf8a_4d36_bf7c_543f95343143.slice/crio-9ed94b0aed50dde7bded52b462a710d66d04ae0ca45af56ed0c00ad6188468e1 WatchSource:0}: Error finding container 9ed94b0aed50dde7bded52b462a710d66d04ae0ca45af56ed0c00ad6188468e1: Status 404 returned error can't find the container with id 9ed94b0aed50dde7bded52b462a710d66d04ae0ca45af56ed0c00ad6188468e1 Jul 16 05:17:33.513444 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.513367 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="313d56a1-8990-49fd-8c06-88b993b58546" path="/var/lib/kubelet/pods/313d56a1-8990-49fd-8c06-88b993b58546/volumes" Jul 16 05:17:33.672707 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.672671 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087259666864" Jul 16 05:17:33.672707 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.672705 2573 scope.go:117] "RemoveContainer" containerID="51ad86711c4724c0519f36a6113d9dad46b93c5d2c5ab84a54764ed7c6ae0a15" Jul 16 05:17:33.674210 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:33.674171 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087261838912" event={"ID":"14ac5079-cf8a-4d36-bf7c-543f95343143","Type":"ContainerStarted","Data":"9ed94b0aed50dde7bded52b462a710d66d04ae0ca45af56ed0c00ad6188468e1"} Jul 16 05:17:34.681319 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:34.681286 2573 generic.go:356] "Generic (PLEG): container finished" podID="14ac5079-cf8a-4d36-bf7c-543f95343143" containerID="3d87acc78cd495bba82bbeff4b32a9e86bc5ac68ad05f0b1d7a48ac30d30256f" exitCode=0 Jul 16 05:17:34.681761 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:34.681371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087261838912" event={"ID":"14ac5079-cf8a-4d36-bf7c-543f95343143","Type":"ContainerDied","Data":"3d87acc78cd495bba82bbeff4b32a9e86bc5ac68ad05f0b1d7a48ac30d30256f"} Jul 16 05:17:35.826079 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.826050 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:35.846164 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.846094 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087261838912_14ac5079-cf8a-4d36-bf7c-543f95343143/test-curl-7491-140087261838912/0.log" Jul 16 05:17:35.868710 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.868672 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:35.874739 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.874711 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:35.974176 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.974068 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4vvc\" (UniqueName: \"kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc\") pod \"14ac5079-cf8a-4d36-bf7c-543f95343143\" (UID: \"14ac5079-cf8a-4d36-bf7c-543f95343143\") " Jul 16 05:17:35.976113 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.976087 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc" (OuterVolumeSpecName: "kube-api-access-d4vvc") pod "14ac5079-cf8a-4d36-bf7c-543f95343143" (UID: "14ac5079-cf8a-4d36-bf7c-543f95343143"). InnerVolumeSpecName "kube-api-access-d4vvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:35.992366 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.992332 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:35.992798 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.992784 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14ac5079-cf8a-4d36-bf7c-543f95343143" containerName="test-curl-7491-140087261838912" Jul 16 05:17:35.992845 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.992800 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="14ac5079-cf8a-4d36-bf7c-543f95343143" containerName="test-curl-7491-140087261838912" Jul 16 05:17:35.992880 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.992865 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="14ac5079-cf8a-4d36-bf7c-543f95343143" containerName="test-curl-7491-140087261838912" Jul 16 05:17:35.995885 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:35.995869 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.002068 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.002036 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:36.075120 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.075083 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lxqv\" (UniqueName: \"kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv\") pod \"test-curl-7491-140087261838912\" (UID: \"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.075325 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.075171 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4vvc\" (UniqueName: \"kubernetes.io/projected/14ac5079-cf8a-4d36-bf7c-543f95343143-kube-api-access-d4vvc\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:36.176262 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.176217 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lxqv\" (UniqueName: \"kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv\") pod \"test-curl-7491-140087261838912\" (UID: \"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.184596 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.184565 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lxqv\" (UniqueName: \"kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv\") pod \"test-curl-7491-140087261838912\" (UID: \"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c\") " pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.306288 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.306180 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.445747 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.445717 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:36.447194 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:17:36.447139 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod159c1ad0_8baa_449a_a4ad_9a5fb1ce9d2c.slice/crio-8f64f578dfcdcf1f961eef03f81000b6a6e1e97ed402f60ea8d32d30e251c243 WatchSource:0}: Error finding container 8f64f578dfcdcf1f961eef03f81000b6a6e1e97ed402f60ea8d32d30e251c243: Status 404 returned error can't find the container with id 8f64f578dfcdcf1f961eef03f81000b6a6e1e97ed402f60ea8d32d30e251c243 Jul 16 05:17:36.695406 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.695368 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:36.695599 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.695378 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ed94b0aed50dde7bded52b462a710d66d04ae0ca45af56ed0c00ad6188468e1" Jul 16 05:17:36.696898 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.696870 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087261838912" event={"ID":"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c","Type":"ContainerStarted","Data":"8f64f578dfcdcf1f961eef03f81000b6a6e1e97ed402f60ea8d32d30e251c243"} Jul 16 05:17:36.707360 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:36.707324 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7491-140087261838912" oldPodUID="14ac5079-cf8a-4d36-bf7c-543f95343143" podUID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" Jul 16 05:17:37.515223 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:37.515188 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14ac5079-cf8a-4d36-bf7c-543f95343143" path="/var/lib/kubelet/pods/14ac5079-cf8a-4d36-bf7c-543f95343143/volumes" Jul 16 05:17:37.703841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:37.703801 2573 generic.go:356] "Generic (PLEG): container finished" podID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" containerID="5d0facac36c06c3124885f0d84c18efd1cfb81e19bec9aa53f981624f472ede0" exitCode=0 Jul 16 05:17:37.704008 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:37.703888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087261838912" event={"ID":"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c","Type":"ContainerDied","Data":"5d0facac36c06c3124885f0d84c18efd1cfb81e19bec9aa53f981624f472ede0"} Jul 16 05:17:38.843636 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:38.843608 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:38.863428 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:38.863402 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087261838912_159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c/test-curl-7491-140087261838912/0.log" Jul 16 05:17:38.886461 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:38.886426 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:38.890640 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:38.890612 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087261838912"] Jul 16 05:17:39.001842 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.001756 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lxqv\" (UniqueName: \"kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv\") pod \"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c\" (UID: \"159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c\") " Jul 16 05:17:39.004301 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.004267 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv" (OuterVolumeSpecName: "kube-api-access-6lxqv") pod "159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" (UID: "159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c"). InnerVolumeSpecName "kube-api-access-6lxqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:39.005738 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.005712 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:39.006085 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.006072 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" containerName="test-curl-7491-140087261838912" Jul 16 05:17:39.006085 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.006085 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" containerName="test-curl-7491-140087261838912" Jul 16 05:17:39.006227 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.006182 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" containerName="test-curl-7491-140087261838912" Jul 16 05:17:39.009022 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.009004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:39.015221 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.015195 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:39.103059 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.103023 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mntx\" (UniqueName: \"kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx\") pod \"test-curl-7491-140087263404656\" (UID: \"953f2678-4a46-42fa-9c46-8003e11ff185\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:39.103273 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.103169 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lxqv\" (UniqueName: \"kubernetes.io/projected/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c-kube-api-access-6lxqv\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:39.203782 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.203744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mntx\" (UniqueName: \"kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx\") pod \"test-curl-7491-140087263404656\" (UID: \"953f2678-4a46-42fa-9c46-8003e11ff185\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:39.212508 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.212476 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mntx\" (UniqueName: \"kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx\") pod \"test-curl-7491-140087263404656\" (UID: \"953f2678-4a46-42fa-9c46-8003e11ff185\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:39.322176 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.322053 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:39.487483 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.487399 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:39.489730 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:17:39.489698 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod953f2678_4a46_42fa_9c46_8003e11ff185.slice/crio-a258706e41fdef63eb79b49fb0dedaf14e8551de24540bfd91da49f0e3162ea1 WatchSource:0}: Error finding container a258706e41fdef63eb79b49fb0dedaf14e8551de24540bfd91da49f0e3162ea1: Status 404 returned error can't find the container with id a258706e41fdef63eb79b49fb0dedaf14e8551de24540bfd91da49f0e3162ea1 Jul 16 05:17:39.520977 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.520936 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c" path="/var/lib/kubelet/pods/159c1ad0-8baa-449a-a4ad-9a5fb1ce9d2c/volumes" Jul 16 05:17:39.717072 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.717036 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087261838912" Jul 16 05:17:39.717072 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.717059 2573 scope.go:117] "RemoveContainer" containerID="5d0facac36c06c3124885f0d84c18efd1cfb81e19bec9aa53f981624f472ede0" Jul 16 05:17:39.718456 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:39.718432 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087263404656" event={"ID":"953f2678-4a46-42fa-9c46-8003e11ff185","Type":"ContainerStarted","Data":"a258706e41fdef63eb79b49fb0dedaf14e8551de24540bfd91da49f0e3162ea1"} Jul 16 05:17:40.727136 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:40.727098 2573 generic.go:356] "Generic (PLEG): container finished" podID="953f2678-4a46-42fa-9c46-8003e11ff185" containerID="6d39af281886715db1f921448d06b48cc713cbca21e146de6230a42e3109a276" exitCode=0 Jul 16 05:17:40.727527 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:40.727243 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087263404656" event={"ID":"953f2678-4a46-42fa-9c46-8003e11ff185","Type":"ContainerDied","Data":"6d39af281886715db1f921448d06b48cc713cbca21e146de6230a42e3109a276"} Jul 16 05:17:41.302496 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.302452 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" podUID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" containerName="main" containerID="cri-o://acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0" gracePeriod=45 Jul 16 05:17:41.563270 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.563208 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:17:41.727642 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727612 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tltkt\" (UniqueName: \"kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727666 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727775 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727803 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727827 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.727852 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm\") pod \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\" (UID: \"39a5f6e3-9252-4d85-9dc8-973cd68f427a\") " Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.728050 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home" (OuterVolumeSpecName: "home") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:41.728121 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.728061 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:41.728553 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.728311 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tmp-dir\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.728553 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.728333 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-home\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.728553 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.728312 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache" (OuterVolumeSpecName: "model-cache") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:41.730363 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.730335 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt" (OuterVolumeSpecName: "kube-api-access-tltkt") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "kube-api-access-tltkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:41.730496 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.730466 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 05:17:41.730558 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.730499 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm" (OuterVolumeSpecName: "dshm") pod "39a5f6e3-9252-4d85-9dc8-973cd68f427a" (UID: "39a5f6e3-9252-4d85-9dc8-973cd68f427a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 05:17:41.735983 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.735955 2573 generic.go:356] "Generic (PLEG): container finished" podID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" containerID="acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0" exitCode=0 Jul 16 05:17:41.736094 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.735990 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" Jul 16 05:17:41.736094 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.736031 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" event={"ID":"39a5f6e3-9252-4d85-9dc8-973cd68f427a","Type":"ContainerDied","Data":"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0"} Jul 16 05:17:41.736094 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.736078 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg" event={"ID":"39a5f6e3-9252-4d85-9dc8-973cd68f427a","Type":"ContainerDied","Data":"081767f7326687a70244cf24ae8f31c95d1bc70a066f41a56c17260af6b9bfa3"} Jul 16 05:17:41.736284 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.736100 2573 scope.go:117] "RemoveContainer" containerID="acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0" Jul 16 05:17:41.752994 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.752971 2573 scope.go:117] "RemoveContainer" containerID="acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0" Jul 16 05:17:41.753341 ip-10-0-137-187 kubenswrapper[2573]: E0716 05:17:41.753314 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0\": container with ID starting with acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0 not found: ID does not exist" containerID="acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0" Jul 16 05:17:41.753431 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.753352 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0"} err="failed to get container status \"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0\": rpc error: code = NotFound desc = could not find container \"acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0\": container with ID starting with acd398a51118c6d31470880c44f18ae3db2dcbe385f14a8948485699becddac0 not found: ID does not exist" Jul 16 05:17:41.763720 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.763697 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:17:41.769421 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.769396 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-12efc5/rate-test-model-12efc5-kserve-595cc7688c-s89sg"] Jul 16 05:17:41.829997 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.829908 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-model-cache\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.829997 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.829945 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39a5f6e3-9252-4d85-9dc8-973cd68f427a-tls-certs\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.829997 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.829957 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39a5f6e3-9252-4d85-9dc8-973cd68f427a-dshm\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.829997 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.829971 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tltkt\" (UniqueName: \"kubernetes.io/projected/39a5f6e3-9252-4d85-9dc8-973cd68f427a-kube-api-access-tltkt\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:41.860988 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.860963 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:41.880467 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.880435 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087263404656_953f2678-4a46-42fa-9c46-8003e11ff185/test-curl-7491-140087263404656/0.log" Jul 16 05:17:41.902575 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.902543 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:41.906247 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:41.906222 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:42.021049 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021013 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:42.021408 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021395 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953f2678-4a46-42fa-9c46-8003e11ff185" containerName="test-curl-7491-140087263404656" Jul 16 05:17:42.021469 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021410 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="953f2678-4a46-42fa-9c46-8003e11ff185" containerName="test-curl-7491-140087263404656" Jul 16 05:17:42.021469 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021420 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" containerName="main" Jul 16 05:17:42.021469 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021425 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" containerName="main" Jul 16 05:17:42.021574 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021495 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="953f2678-4a46-42fa-9c46-8003e11ff185" containerName="test-curl-7491-140087263404656" Jul 16 05:17:42.021574 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.021515 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" containerName="main" Jul 16 05:17:42.024538 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.024519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.030520 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.030484 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:42.031553 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.031526 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mntx\" (UniqueName: \"kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx\") pod \"953f2678-4a46-42fa-9c46-8003e11ff185\" (UID: \"953f2678-4a46-42fa-9c46-8003e11ff185\") " Jul 16 05:17:42.033969 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.033943 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx" (OuterVolumeSpecName: "kube-api-access-2mntx") pod "953f2678-4a46-42fa-9c46-8003e11ff185" (UID: "953f2678-4a46-42fa-9c46-8003e11ff185"). InnerVolumeSpecName "kube-api-access-2mntx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:42.132530 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.132499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfvbw\" (UniqueName: \"kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw\") pod \"test-curl-7491-140087263404656\" (UID: \"6ad1202f-091d-4e24-b1e4-b7413f9f37ab\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.132710 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.132620 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mntx\" (UniqueName: \"kubernetes.io/projected/953f2678-4a46-42fa-9c46-8003e11ff185-kube-api-access-2mntx\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:42.233140 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.233106 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfvbw\" (UniqueName: \"kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw\") pod \"test-curl-7491-140087263404656\" (UID: \"6ad1202f-091d-4e24-b1e4-b7413f9f37ab\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.244024 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.243997 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfvbw\" (UniqueName: \"kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw\") pod \"test-curl-7491-140087263404656\" (UID: \"6ad1202f-091d-4e24-b1e4-b7413f9f37ab\") " pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.335366 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.335336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.470474 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.470450 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:42.472745 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:17:42.472712 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ad1202f_091d_4e24_b1e4_b7413f9f37ab.slice/crio-70f79001826c7b6a05a3b3faed591fbdb6b8ebc0e6b2472b5f4a9c4ba162a8e4 WatchSource:0}: Error finding container 70f79001826c7b6a05a3b3faed591fbdb6b8ebc0e6b2472b5f4a9c4ba162a8e4: Status 404 returned error can't find the container with id 70f79001826c7b6a05a3b3faed591fbdb6b8ebc0e6b2472b5f4a9c4ba162a8e4 Jul 16 05:17:42.742276 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.742191 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:42.742276 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.742211 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a258706e41fdef63eb79b49fb0dedaf14e8551de24540bfd91da49f0e3162ea1" Jul 16 05:17:42.743510 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.743487 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087263404656" event={"ID":"6ad1202f-091d-4e24-b1e4-b7413f9f37ab","Type":"ContainerStarted","Data":"70f79001826c7b6a05a3b3faed591fbdb6b8ebc0e6b2472b5f4a9c4ba162a8e4"} Jul 16 05:17:42.754384 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:42.754361 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7491-140087263404656" oldPodUID="953f2678-4a46-42fa-9c46-8003e11ff185" podUID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" Jul 16 05:17:43.514468 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:43.514434 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a5f6e3-9252-4d85-9dc8-973cd68f427a" path="/var/lib/kubelet/pods/39a5f6e3-9252-4d85-9dc8-973cd68f427a/volumes" Jul 16 05:17:43.518163 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:43.515351 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953f2678-4a46-42fa-9c46-8003e11ff185" path="/var/lib/kubelet/pods/953f2678-4a46-42fa-9c46-8003e11ff185/volumes" Jul 16 05:17:43.751761 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:43.751721 2573 generic.go:356] "Generic (PLEG): container finished" podID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" containerID="685316586e26b3aec2280660832fbf24372a20450f77b14093f7dad126206ae5" exitCode=0 Jul 16 05:17:43.752175 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:43.751809 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140087263404656" event={"ID":"6ad1202f-091d-4e24-b1e4-b7413f9f37ab","Type":"ContainerDied","Data":"685316586e26b3aec2280660832fbf24372a20450f77b14093f7dad126206ae5"} Jul 16 05:17:44.895048 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:44.895016 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:44.915815 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:44.915765 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140087263404656_6ad1202f-091d-4e24-b1e4-b7413f9f37ab/test-curl-7491-140087263404656/0.log" Jul 16 05:17:44.937225 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:44.937192 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:44.941215 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:44.941192 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140087263404656"] Jul 16 05:17:45.059470 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.059375 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfvbw\" (UniqueName: \"kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw\") pod \"6ad1202f-091d-4e24-b1e4-b7413f9f37ab\" (UID: \"6ad1202f-091d-4e24-b1e4-b7413f9f37ab\") " Jul 16 05:17:45.061481 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.061459 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw" (OuterVolumeSpecName: "kube-api-access-bfvbw") pod "6ad1202f-091d-4e24-b1e4-b7413f9f37ab" (UID: "6ad1202f-091d-4e24-b1e4-b7413f9f37ab"). InnerVolumeSpecName "kube-api-access-bfvbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 05:17:45.160671 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.160637 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfvbw\" (UniqueName: \"kubernetes.io/projected/6ad1202f-091d-4e24-b1e4-b7413f9f37ab-kube-api-access-bfvbw\") on node \"ip-10-0-137-187.ec2.internal\" DevicePath \"\"" Jul 16 05:17:45.513176 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.513123 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" path="/var/lib/kubelet/pods/6ad1202f-091d-4e24-b1e4-b7413f9f37ab/volumes" Jul 16 05:17:45.763582 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.763498 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140087263404656" Jul 16 05:17:45.763582 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:17:45.763521 2573 scope.go:117] "RemoveContainer" containerID="685316586e26b3aec2280660832fbf24372a20450f77b14093f7dad126206ae5" Jul 16 05:18:17.928414 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:17.928383 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-nj4sn_1a2d8752-4130-44f7-bd13-e3d3867532d2/copy-manifests/0.log" Jul 16 05:18:17.939172 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:17.939128 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-nj4sn_1a2d8752-4130-44f7-bd13-e3d3867532d2/manager/0.log" Jul 16 05:18:19.473391 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.473358 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/util/0.log" Jul 16 05:18:19.480475 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.480453 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/pull/0.log" Jul 16 05:18:19.487039 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.487013 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/extract/0.log" Jul 16 05:18:19.608902 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.608878 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/util/0.log" Jul 16 05:18:19.615489 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.615463 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/pull/0.log" Jul 16 05:18:19.624100 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.624081 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/extract/0.log" Jul 16 05:18:19.748732 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.748667 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/util/0.log" Jul 16 05:18:19.756188 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.756164 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/pull/0.log" Jul 16 05:18:19.763395 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.763370 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/extract/0.log" Jul 16 05:18:19.890610 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.890569 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/pull/0.log" Jul 16 05:18:19.898252 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.898225 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/extract/0.log" Jul 16 05:18:19.905418 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:19.905393 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/util/0.log" Jul 16 05:18:20.717872 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:20.717839 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6mvgw_9b0bc6c5-5080-433f-93f5-8946b5cba5cd/manager/0.log" Jul 16 05:18:21.605378 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:21.605349 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-dcxmr_4349e764-9aa9-4fb3-b719-1eeb64a25970/kube-auth-proxy/0.log" Jul 16 05:18:21.972009 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:21.971924 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-frht7_5262d330-1b50-440e-81bb-fea35169d344/payload-pre-processing/0.log" Jul 16 05:18:22.578922 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:22.578887 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6dnjz_9f96c3de-a33c-47f5-9875-58322f9f6c96/main/0.log" Jul 16 05:18:22.923014 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:22.922988 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gr6h_4f4fc37f-91c3-47c2-b389-2d09a2a76029/main/0.log" Jul 16 05:18:23.184277 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:23.184181 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-rsjrj_b3931727-11e8-4529-9b19-0b90f25508d9/main/0.log" Jul 16 05:18:28.853025 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.852990 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5c2z4/must-gather-bjmmc"] Jul 16 05:18:28.853587 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.853416 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" containerName="test-curl-7491-140087263404656" Jul 16 05:18:28.853587 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.853438 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" containerName="test-curl-7491-140087263404656" Jul 16 05:18:28.853587 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.853534 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad1202f-091d-4e24-b1e4-b7413f9f37ab" containerName="test-curl-7491-140087263404656" Jul 16 05:18:28.857260 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.857239 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:28.860600 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.860576 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5c2z4\"/\"default-dockercfg-jr99c\"" Jul 16 05:18:28.861440 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.861414 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5c2z4\"/\"openshift-service-ca.crt\"" Jul 16 05:18:28.861546 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.861473 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5c2z4\"/\"kube-root-ca.crt\"" Jul 16 05:18:28.868453 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.868430 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/must-gather-bjmmc"] Jul 16 05:18:28.962273 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.962242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn7qx\" (UniqueName: \"kubernetes.io/projected/9c2fbbf9-f690-4e95-8425-08f331c21908-kube-api-access-zn7qx\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:28.962467 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:28.962302 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9c2fbbf9-f690-4e95-8425-08f331c21908-must-gather-output\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.063367 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.063336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn7qx\" (UniqueName: \"kubernetes.io/projected/9c2fbbf9-f690-4e95-8425-08f331c21908-kube-api-access-zn7qx\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.063561 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.063410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9c2fbbf9-f690-4e95-8425-08f331c21908-must-gather-output\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.063786 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.063762 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9c2fbbf9-f690-4e95-8425-08f331c21908-must-gather-output\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.073293 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.073271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn7qx\" (UniqueName: \"kubernetes.io/projected/9c2fbbf9-f690-4e95-8425-08f331c21908-kube-api-access-zn7qx\") pod \"must-gather-bjmmc\" (UID: \"9c2fbbf9-f690-4e95-8425-08f331c21908\") " pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.167781 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.167748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" Jul 16 05:18:29.303939 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:29.303910 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/must-gather-bjmmc"] Jul 16 05:18:29.305238 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:18:29.305205 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c2fbbf9_f690_4e95_8425_08f331c21908.slice/crio-86a128cb8689e9cf52c3d6d809445f30f4c196dd347e0907e075935d806bd990 WatchSource:0}: Error finding container 86a128cb8689e9cf52c3d6d809445f30f4c196dd347e0907e075935d806bd990: Status 404 returned error can't find the container with id 86a128cb8689e9cf52c3d6d809445f30f4c196dd347e0907e075935d806bd990 Jul 16 05:18:30.004221 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:30.004178 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" event={"ID":"9c2fbbf9-f690-4e95-8425-08f331c21908","Type":"ContainerStarted","Data":"86a128cb8689e9cf52c3d6d809445f30f4c196dd347e0907e075935d806bd990"} Jul 16 05:18:31.015508 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:31.015454 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" event={"ID":"9c2fbbf9-f690-4e95-8425-08f331c21908","Type":"ContainerStarted","Data":"dbe6c61fdb71214e54c517c75114e91d7e7042b55622d45961a1d3057e394f68"} Jul 16 05:18:31.015508 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:31.015514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" event={"ID":"9c2fbbf9-f690-4e95-8425-08f331c21908","Type":"ContainerStarted","Data":"efe74d6590c77923a36f6e57796787bd66ed585e6c8258a443aaca91ea9390c7"} Jul 16 05:18:31.035245 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:31.035176 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5c2z4/must-gather-bjmmc" podStartSLOduration=2.063382934 podStartE2EDuration="3.03513723s" totalImagesPullingTime="971.754296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 05:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:18:31.034840909 +0000 UTC m=+2374.761211210" watchObservedRunningTime="2026-07-16 05:18:31.03513723 +0000 UTC m=+2374.761507531" Jul 16 05:18:31.957242 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:31.957212 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vpthh_b3817b1e-22ed-4fe3-b28b-3b76bfc8bf79/global-pull-secret-syncer/0.log" Jul 16 05:18:32.114489 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:32.114453 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tq4p6_dd48f9f6-380b-4ab0-b5b1-02341b0d9801/konnectivity-agent/0.log" Jul 16 05:18:32.164056 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:32.164025 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-187.ec2.internal_c4a62ec9b3b57c8a82085d965555b5bf/haproxy/0.log" Jul 16 05:18:36.750265 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.750187 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/extract/0.log" Jul 16 05:18:36.783139 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.783062 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/util/0.log" Jul 16 05:18:36.811007 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.810943 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759zg6kj_465b14e0-d9a4-400a-8080-d039f504a226/pull/0.log" Jul 16 05:18:36.843911 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.843886 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/extract/0.log" Jul 16 05:18:36.871204 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.871106 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/util/0.log" Jul 16 05:18:36.894025 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.893998 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c6b87_070c9160-66f4-4924-ba24-06001e31b5fa/pull/0.log" Jul 16 05:18:36.921854 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.921827 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/extract/0.log" Jul 16 05:18:36.943684 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.943651 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/util/0.log" Jul 16 05:18:36.967841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:36.967807 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wrcdg_295dfdc2-b7cd-4fff-b663-69392aa1fac1/pull/0.log" Jul 16 05:18:37.022923 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:37.022840 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/extract/0.log" Jul 16 05:18:37.047274 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:37.047247 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/util/0.log" Jul 16 05:18:37.079018 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:37.078987 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12hmv2_48054b49-7800-43a6-ab7e-0eec5a76f2c2/pull/0.log" Jul 16 05:18:37.644933 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:37.644895 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6mvgw_9b0bc6c5-5080-433f-93f5-8946b5cba5cd/manager/0.log" Jul 16 05:18:39.179671 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.179636 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/alertmanager/0.log" Jul 16 05:18:39.203020 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.202989 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/config-reloader/0.log" Jul 16 05:18:39.225841 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.225812 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/kube-rbac-proxy-web/0.log" Jul 16 05:18:39.251406 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.251376 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/kube-rbac-proxy/0.log" Jul 16 05:18:39.274662 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.274634 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/kube-rbac-proxy-metric/0.log" Jul 16 05:18:39.302790 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.302763 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/prom-label-proxy/0.log" Jul 16 05:18:39.329877 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.329842 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_006823b2-d704-4b73-a596-4b537e43de6a/init-config-reloader/0.log" Jul 16 05:18:39.399412 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.399378 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ml2rb_3aaabf2f-3945-4ca3-b89c-34017bdd555e/kube-state-metrics/0.log" Jul 16 05:18:39.418794 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.418760 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ml2rb_3aaabf2f-3945-4ca3-b89c-34017bdd555e/kube-rbac-proxy-main/0.log" Jul 16 05:18:39.451862 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.451831 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ml2rb_3aaabf2f-3945-4ca3-b89c-34017bdd555e/kube-rbac-proxy-self/0.log" Jul 16 05:18:39.487734 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.487706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-688d44c87-fp6dc_d21509a6-dfb9-4919-b473-1c90aa53c908/metrics-server/0.log" Jul 16 05:18:39.711300 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.711213 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s4r7b_b03d00f9-36c7-434c-8fa9-88e00d28ae55/node-exporter/0.log" Jul 16 05:18:39.733928 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.733896 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s4r7b_b03d00f9-36c7-434c-8fa9-88e00d28ae55/kube-rbac-proxy/0.log" Jul 16 05:18:39.757258 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.757227 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s4r7b_b03d00f9-36c7-434c-8fa9-88e00d28ae55/init-textfile/0.log" Jul 16 05:18:39.786126 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.786088 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-l8kg8_98839d4e-bdf3-4e2a-9dde-b1b8be065bed/kube-rbac-proxy-main/0.log" Jul 16 05:18:39.808486 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.808452 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-l8kg8_98839d4e-bdf3-4e2a-9dde-b1b8be065bed/kube-rbac-proxy-self/0.log" Jul 16 05:18:39.832967 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:39.832936 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-l8kg8_98839d4e-bdf3-4e2a-9dde-b1b8be065bed/openshift-state-metrics/0.log" Jul 16 05:18:40.093431 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:40.093346 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-mjwp5_e0bbf9e5-927b-4455-8a0c-c400b5db2b30/prometheus-operator-admission-webhook/0.log" Jul 16 05:18:40.130883 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:40.130846 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7ffd7cf8b5-rr2zs_af7efca5-9d0b-498b-afe8-b6ced15f7ac3/telemeter-client/0.log" Jul 16 05:18:40.157218 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:40.157189 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7ffd7cf8b5-rr2zs_af7efca5-9d0b-498b-afe8-b6ced15f7ac3/reload/0.log" Jul 16 05:18:40.180287 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:40.180259 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7ffd7cf8b5-rr2zs_af7efca5-9d0b-498b-afe8-b6ced15f7ac3/kube-rbac-proxy/0.log" Jul 16 05:18:41.982995 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:41.982961 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n"] Jul 16 05:18:41.991643 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:41.991615 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:41.997801 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:41.997778 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n"] Jul 16 05:18:42.099038 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.099000 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-lib-modules\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.105077 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.105046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-sys\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.105377 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.105360 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-podres\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.105621 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.105605 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrr4\" (UniqueName: \"kubernetes.io/projected/81f75a57-418c-4bf3-94a5-5a97594680fe-kube-api-access-rbrr4\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.105750 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.105738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-proc\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.206709 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.206676 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-podres\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.206925 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.206899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrr4\" (UniqueName: \"kubernetes.io/projected/81f75a57-418c-4bf3-94a5-5a97594680fe-kube-api-access-rbrr4\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207018 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.206939 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-proc\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207018 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.206981 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-lib-modules\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207119 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.207026 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-sys\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207119 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.207025 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-podres\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207251 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.207173 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-sys\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207316 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.207281 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-lib-modules\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.207486 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.207461 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81f75a57-418c-4bf3-94a5-5a97594680fe-proc\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.216371 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.216344 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrr4\" (UniqueName: \"kubernetes.io/projected/81f75a57-418c-4bf3-94a5-5a97594680fe-kube-api-access-rbrr4\") pod \"perf-node-gather-daemonset-fq52n\" (UID: \"81f75a57-418c-4bf3-94a5-5a97594680fe\") " pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.306435 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.306059 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:42.489480 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.489136 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n"] Jul 16 05:18:42.493881 ip-10-0-137-187 kubenswrapper[2573]: W0716 05:18:42.493848 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81f75a57_418c_4bf3_94a5_5a97594680fe.slice/crio-aeed0217a4ce2eb289b9583f0530fb53199e1e2ad019d70c982f7578ad9cfa06 WatchSource:0}: Error finding container aeed0217a4ce2eb289b9583f0530fb53199e1e2ad019d70c982f7578ad9cfa06: Status 404 returned error can't find the container with id aeed0217a4ce2eb289b9583f0530fb53199e1e2ad019d70c982f7578ad9cfa06 Jul 16 05:18:42.778812 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:42.778774 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-566696f4dd-sfrmk_547e5b15-2c4b-48cf-bc4b-54b8825faef5/console/0.log" Jul 16 05:18:43.128855 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:43.128815 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" event={"ID":"81f75a57-418c-4bf3-94a5-5a97594680fe","Type":"ContainerStarted","Data":"cc3b7e5de290cf041377329926d09c2ab08e6cee9a30fcaadced7b55759f643e"} Jul 16 05:18:43.129398 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:43.128918 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:43.129398 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:43.128945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" event={"ID":"81f75a57-418c-4bf3-94a5-5a97594680fe","Type":"ContainerStarted","Data":"aeed0217a4ce2eb289b9583f0530fb53199e1e2ad019d70c982f7578ad9cfa06"} Jul 16 05:18:43.158942 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:43.158872 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" podStartSLOduration=2.158850474 podStartE2EDuration="2.158850474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 05:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 05:18:43.155035677 +0000 UTC m=+2386.881405980" watchObservedRunningTime="2026-07-16 05:18:43.158850474 +0000 UTC m=+2386.885220776" Jul 16 05:18:44.589936 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:44.589895 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57lc4_5648f2ca-69e3-4286-9af0-ef623a7192bd/dns/0.log" Jul 16 05:18:44.615302 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:44.615273 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57lc4_5648f2ca-69e3-4286-9af0-ef623a7192bd/kube-rbac-proxy/0.log" Jul 16 05:18:44.741129 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:44.741099 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7rqwt_c3a26870-a25b-474e-a245-ba01fcd77b05/dns-node-resolver/0.log" Jul 16 05:18:45.494241 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:45.494213 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgbzl_053d5cd9-8f78-4549-bdcc-5139d8b44bd1/node-ca/0.log" Jul 16 05:18:46.582105 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:46.582073 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-dcxmr_4349e764-9aa9-4fb3-b719-1eeb64a25970/kube-auth-proxy/0.log" Jul 16 05:18:46.748215 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:46.748177 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-frht7_5262d330-1b50-440e-81bb-fea35169d344/payload-pre-processing/0.log" Jul 16 05:18:47.551184 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:47.551139 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v82sc_5d722f5c-1317-4c46-aee0-ccabafad67a2/serve-healthcheck-canary/0.log" Jul 16 05:18:48.124485 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:48.124455 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jh6mv_d12b3c0d-7c5f-44fb-a91c-a1cdff705271/kube-rbac-proxy/0.log" Jul 16 05:18:48.146465 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:48.146421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jh6mv_d12b3c0d-7c5f-44fb-a91c-a1cdff705271/exporter/0.log" Jul 16 05:18:48.168068 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:48.168034 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jh6mv_d12b3c0d-7c5f-44fb-a91c-a1cdff705271/extractor/0.log" Jul 16 05:18:49.148987 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:49.148960 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5c2z4/perf-node-gather-daemonset-fq52n" Jul 16 05:18:50.907922 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:50.907870 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-nj4sn_1a2d8752-4130-44f7-bd13-e3d3867532d2/manager/0.log" Jul 16 05:18:50.933800 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:50.933774 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b4b7cb559-nj4sn_1a2d8752-4130-44f7-bd13-e3d3867532d2/copy-manifests/0.log" Jul 16 05:18:52.344329 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:52.344245 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-546f8cbbbf-zxcbz_f0b2c0d7-cd10-4e46-b679-58bb1881b202/manager/0.log" Jul 16 05:18:52.744396 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:52.744360 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-dcxmr_4349e764-9aa9-4fb3-b719-1eeb64a25970/kube-auth-proxy/0.log" Jul 16 05:18:52.846114 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:52.846084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-frht7_5262d330-1b50-440e-81bb-fea35169d344/payload-pre-processing/0.log" Jul 16 05:18:54.228691 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:54.228607 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68764b5749-dcxmr_4349e764-9aa9-4fb3-b719-1eeb64a25970/kube-auth-proxy/0.log" Jul 16 05:18:54.323283 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:54.323251 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-frht7_5262d330-1b50-440e-81bb-fea35169d344/payload-pre-processing/0.log" Jul 16 05:18:57.014882 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:57.014853 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:18:57.022919 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:57.022888 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7c4d9_d32e28d9-9cde-4522-af4b-bac6b04ab850/ovn-acl-logging/0.log" Jul 16 05:18:57.387187 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:57.387136 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-gsxmz_b2868b05-91d9-423c-8ffc-c2e9ff73734a/migrator/0.log" Jul 16 05:18:57.407975 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:57.407949 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-gsxmz_b2868b05-91d9-423c-8ffc-c2e9ff73734a/graceful-termination/0.log" Jul 16 05:18:58.927123 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:58.927093 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/kube-multus-additional-cni-plugins/0.log" Jul 16 05:18:58.948839 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:58.948791 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/egress-router-binary-copy/0.log" Jul 16 05:18:58.969704 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:58.969674 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/cni-plugins/0.log" Jul 16 05:18:58.990326 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:58.990293 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/bond-cni-plugin/0.log" Jul 16 05:18:59.013862 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.013837 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/routeoverride-cni/0.log" Jul 16 05:18:59.035087 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.035063 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/whereabouts-cni-bincopy/0.log" Jul 16 05:18:59.055705 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.055674 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j8gh6_ab0b8cd2-0e23-41ac-b183-41b735183d6e/whereabouts-cni/0.log" Jul 16 05:18:59.286932 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.286852 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zpjzr_db73ec87-38ea-4629-a015-bdb64c988d73/kube-multus/0.log" Jul 16 05:18:59.360731 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.360701 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccprx_03421519-f85c-4e08-aa29-5b1f437f72bf/network-metrics-daemon/0.log" Jul 16 05:18:59.378946 ip-10-0-137-187 kubenswrapper[2573]: I0716 05:18:59.378918 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccprx_03421519-f85c-4e08-aa29-5b1f437f72bf/kube-rbac-proxy/0.log"