Jul 08 18:02:29.444753 ip-10-0-131-126 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:02:29.444768 ip-10-0-131-126 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:02:29.444778 ip-10-0-131-126 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:02:29.445110 ip-10-0-131-126 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:02:39.693833 ip-10-0-131-126 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 18:02:39.693850 ip-10-0-131-126 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 49c4f0f317be40d380452ea40c76833d -- Jul 08 18:04:45.540733 ip-10-0-131-126 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:04:45.936898 ip-10-0-131-126 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:45.936898 ip-10-0-131-126 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:04:45.936898 ip-10-0-131-126 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:45.936898 ip-10-0-131-126 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:04:45.936898 ip-10-0-131-126 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:04:45.938023 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.937946 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:04:45.941322 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941306 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941323 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941327 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941330 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941334 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941336 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941339 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941342 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941344 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941347 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941349 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941352 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941354 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941357 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941360 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941363 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941365 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941368 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.941361 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941370 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941373 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941376 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941378 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941383 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941386 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941390 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941399 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941402 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941404 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941407 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941409 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941412 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941414 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941417 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941419 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941422 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941424 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941427 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.941809 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941429 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941431 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941434 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941436 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941438 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941441 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941443 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941446 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941448 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941450 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941465 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941468 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941471 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941473 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941476 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941479 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941482 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941485 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941487 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941490 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.942298 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941493 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941496 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941500 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941503 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941506 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941509 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941512 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941514 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941517 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941520 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941522 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941525 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941528 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941530 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941533 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941535 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941537 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941540 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941543 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941545 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.942814 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941547 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941550 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941553 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941555 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941558 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941560 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941565 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941570 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941573 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941576 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941579 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941581 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941584 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941587 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941589 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941592 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941595 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941597 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941600 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.943297 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941603 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941605 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.941608 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942277 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942283 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942286 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942288 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942292 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942294 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942296 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942299 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942302 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942304 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942308 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942310 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942313 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942316 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942319 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942321 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942324 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.943766 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942327 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942330 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942332 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942335 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942337 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942340 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942342 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942347 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942351 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942354 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942357 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942360 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942363 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942366 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942368 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942371 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942373 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942376 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942378 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.944262 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942381 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942383 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942386 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942388 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942391 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942393 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942395 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942398 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942400 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942403 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942405 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942408 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942411 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942414 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942417 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942419 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942422 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942425 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942427 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942431 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.944746 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942433 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942436 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942439 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942441 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942444 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942447 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942449 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942469 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942472 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942475 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942477 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942480 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942483 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942501 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942505 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942507 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942510 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942512 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942515 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942517 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.945226 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942520 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942522 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942525 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942527 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942531 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942534 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942537 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942539 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942542 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942544 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942547 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942550 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942554 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942558 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942560 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942563 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942565 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942567 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942570 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942572 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.945722 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942575 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942578 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942580 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942937 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942942 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942945 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942948 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942950 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942953 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942955 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942958 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942961 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942963 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942965 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942970 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942973 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942977 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942981 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942983 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.946201 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942986 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942989 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942991 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942994 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942996 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.942999 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943015 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943018 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943021 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943024 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943027 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943031 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943034 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943037 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943039 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943042 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943045 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943047 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943050 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.946686 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943054 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943058 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943060 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943063 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943065 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943068 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943070 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943073 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943075 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943078 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943081 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943085 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943088 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943090 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943093 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943095 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943098 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943100 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943102 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943105 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.947146 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943107 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943110 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943112 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943115 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943117 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943119 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943121 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943124 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943126 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943128 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943131 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943133 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943135 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943140 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943142 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943145 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943147 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943149 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943152 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943154 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.947693 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943157 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943159 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943161 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943164 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943167 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943169 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943172 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943175 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943178 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943180 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943182 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943185 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943187 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943190 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943192 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943194 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943197 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943199 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943202 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943204 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.948180 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943214 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943217 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943219 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943222 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943287 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943295 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943302 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943306 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943310 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943313 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943318 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943321 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943324 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943327 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943330 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943334 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943337 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943340 2562 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943343 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943346 2562 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943349 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943352 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943356 2562 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943359 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:04:45.948672 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943361 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943365 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943368 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943371 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943374 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943377 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943380 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943383 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943386 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943389 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943392 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943395 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943399 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943403 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943405 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943408 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943411 2562 flags.go:64] FLAG: --enable-server="true" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943413 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943418 2562 flags.go:64] FLAG: --event-burst="100" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943420 2562 flags.go:64] FLAG: --event-qps="50" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943423 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943426 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943429 2562 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943434 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:04:45.949414 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943437 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943439 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943443 2562 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943447 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943449 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943466 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943469 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943472 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943475 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943477 2562 flags.go:64] FLAG: --feature-gates="" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943481 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943484 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943487 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943490 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943493 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943496 2562 flags.go:64] FLAG: --help="false" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943498 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-131-126.ec2.internal" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943502 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943504 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943507 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943510 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943515 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943517 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:04:45.950022 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943520 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943522 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943525 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943528 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943531 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943534 2562 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943536 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943539 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943544 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943547 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943550 2562 flags.go:64] FLAG: --lock-file="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943552 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943555 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943559 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943564 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943567 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943569 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943572 2562 flags.go:64] FLAG: --logging-format="text" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943575 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943578 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943581 2562 flags.go:64] FLAG: --manifest-url="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943583 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943587 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943590 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943593 2562 flags.go:64] FLAG: --max-pods="110" Jul 08 18:04:45.950658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943596 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943599 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943601 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943604 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943607 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943611 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943614 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943620 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943623 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943626 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943628 2562 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943631 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943636 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943638 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943641 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943645 2562 flags.go:64] FLAG: --port="10250" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943648 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943651 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0970a0e5d7c2b672b" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943654 2562 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943657 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943660 2562 flags.go:64] FLAG: --register-node="true" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943663 2562 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943666 2562 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943669 2562 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:04:45.951298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943672 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943675 2562 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943678 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943681 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943684 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943686 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943689 2562 flags.go:64] FLAG: --runonce="false" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943691 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943694 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943697 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943700 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943702 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943705 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943708 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943711 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943714 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943717 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943720 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943722 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943725 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943728 2562 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943730 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943735 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943740 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943743 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:04:45.951913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943746 2562 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943749 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943751 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943754 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943757 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943760 2562 flags.go:64] FLAG: --v="2" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943763 2562 flags.go:64] FLAG: --version="false" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943767 2562 flags.go:64] FLAG: --vmodule="" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943771 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.943774 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943884 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943889 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943892 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943895 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943897 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943900 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943902 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943905 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943907 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943909 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943912 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943915 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.952545 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943918 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943920 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943923 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943925 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943929 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943932 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943935 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943938 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943942 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943944 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943947 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943949 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943951 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943954 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943956 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943959 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943962 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943964 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943966 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.953111 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943969 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943972 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943976 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943979 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943981 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943984 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943986 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943989 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943992 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943994 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943997 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.943999 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944002 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944006 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944009 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944011 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944013 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944016 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944018 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.953588 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944021 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944023 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944027 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944029 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944032 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944034 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944036 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944039 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944041 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944044 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944046 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944049 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944051 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944053 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944056 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944059 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944062 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944064 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944067 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944070 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.954086 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944072 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944074 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944077 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944079 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944082 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944084 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944086 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944089 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944091 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944094 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944096 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944098 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944101 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944103 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944106 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944110 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944113 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944116 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944119 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.954610 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944121 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944124 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944126 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944128 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944131 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944134 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944136 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944139 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944141 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.944143 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.944151 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.950409 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.950423 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.950443 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.950464 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:04:45.955084 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950535 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950540 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950544 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950548 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950552 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950554 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950557 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950560 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950563 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950565 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950567 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950570 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950573 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950575 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950578 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950580 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950582 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950585 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950587 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950590 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.955491 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950592 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950595 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950597 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950600 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950602 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950605 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950607 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950609 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950612 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950616 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950618 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950621 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950623 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950626 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950628 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950631 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950633 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950637 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950641 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950644 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.955983 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950647 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950650 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950653 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950655 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950657 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950660 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950662 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950665 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950668 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950670 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950674 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950678 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950681 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950683 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950686 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950689 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950692 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950695 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950697 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.956548 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950700 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950702 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950706 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950710 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950712 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950715 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950718 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950720 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950723 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950726 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950728 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950731 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950734 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950736 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950739 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950741 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950744 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950747 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950750 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950752 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.957004 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950755 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950757 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950760 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950762 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950765 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950767 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950770 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950772 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950774 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950777 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950779 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950781 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950784 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950786 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950789 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950791 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950794 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950797 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950799 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.957636 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950801 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.950806 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950930 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950934 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950937 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950940 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950942 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950945 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950948 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950950 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950953 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950955 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950958 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950960 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:04:45.958407 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950963 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950965 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950968 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950970 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950972 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950975 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950977 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950980 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950982 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950985 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950988 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950991 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950994 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.950997 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951001 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951003 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951007 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951009 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951012 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951014 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:04:45.958933 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951017 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951019 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951022 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951024 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951027 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951029 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951031 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951033 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951036 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951038 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951040 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951042 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951045 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951047 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951049 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951052 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951055 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951059 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951061 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951064 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:04:45.959528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951066 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951069 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951072 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951074 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951076 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951079 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951082 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951084 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951087 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951089 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951092 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951094 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951096 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951099 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951101 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951104 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951106 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951108 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951111 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951113 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:04:45.960007 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951115 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951118 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951120 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951122 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951125 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951127 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951129 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951132 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951134 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951136 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951139 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951141 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951143 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951145 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951148 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951150 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951152 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951155 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951158 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951160 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:04:45.960544 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951163 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951165 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951170 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951174 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951177 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951180 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:45.951183 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.951188 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.951292 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.953963 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.955304 2562 server.go:1019] "Starting client certificate rotation" Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.955408 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.955434 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:04:45.961025 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.959652 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:04:45.962490 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.962475 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:04:45.973420 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.973402 2562 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:04:45.975222 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.975199 2562 log.go:25] "Validated CRI v1 image API" Jul 08 18:04:45.975983 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.975972 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:04:45.978397 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.978372 2562 fs.go:135] Filesystem UUIDs: map[3f5bcd21-bf6d-43ca-8d61-45b4da064db3:/dev/nvme0n1p4 48545ee7-08b7-4e46-87f4-b24cec25c720:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 08 18:04:45.978495 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.978396 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:04:45.982298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.982280 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:04:45.983550 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.983421 2562 manager.go:216] Machine: {Timestamp:2026-07-08 18:04:45.982093319 +0000 UTC m=+0.321537319 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105054 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29cbe5861cde9b864745342d57774d SystemUUID:ec29cbe5-861c-de9b-8647-45342d57774d BootID:49c4f0f3-17be-40d3-8045-2ea40c76833d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:2e:79:6b:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:2e:79:6b:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:f1:a4:53:19:a1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:04:45.983550 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.983546 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:04:45.983699 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.983648 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:04:45.986880 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.986858 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:04:45.987011 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.986884 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-126.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:04:45.987551 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.987541 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:04:45.987582 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.987553 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:04:45.987582 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.987560 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:04:45.988156 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.988146 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:04:45.989578 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.989568 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:04:45.993234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993222 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:04:45.993347 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993338 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:04:45.993385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993370 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:04:45.993385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993379 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:04:45.993486 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993393 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:04:45.993486 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993403 2562 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:04:45.993486 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993412 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:04:45.993791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993780 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:04:45.993825 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.993798 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:04:45.996937 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.996916 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:04:45.998668 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.998653 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:04:45.998741 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.998684 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:04:45.999603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999585 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:04:45.999603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999602 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:04:45.999603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999608 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999613 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999622 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999627 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999634 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999639 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999646 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999651 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999659 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:04:45.999739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:45.999664 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:04:46.000313 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.000304 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:04:46.000313 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.000313 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:04:46.003433 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.003418 2562 server.go:1268] "Started kubelet" Jul 08 18:04:46.003532 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.003510 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:04:46.003649 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.003610 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:04:46.003711 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.003666 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:04:46.004325 ip-10-0-131-126 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:04:46.005564 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.005543 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-126.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:04:46.005739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.005726 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:04:46.006003 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.005982 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:04:46.006068 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.005988 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:04:46.007571 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.007552 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:04:46.008614 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.008594 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:04:46.011674 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.010792 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea81aa2d7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.003389143 +0000 UTC m=+0.342833142,LastTimestamp:2026-07-08 18:04:46.003389143 +0000 UTC m=+0.342833142,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.011859 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.011842 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:04:46.011943 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.011862 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:04:46.011943 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.011932 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.012345 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012325 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:04:46.012435 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012391 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:04:46.012435 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012405 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:04:46.012559 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012531 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:04:46.012559 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012540 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:04:46.012643 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012597 2562 factory.go:55] Registering systemd factory Jul 08 18:04:46.012643 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012611 2562 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:04:46.012868 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012849 2562 factory.go:153] Registering CRI-O factory Jul 08 18:04:46.012868 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012868 2562 factory.go:223] Registration of the crio container factory successfully Jul 08 18:04:46.012980 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012924 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:04:46.012980 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012941 2562 factory.go:103] Registering Raw factory Jul 08 18:04:46.012980 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.012951 2562 manager.go:1196] Started watching for new ooms in manager Jul 08 18:04:46.013631 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.013615 2562 manager.go:349] Starting recovery of all containers Jul 08 18:04:46.020552 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.020501 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:04:46.023335 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.023310 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 18:04:46.023432 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.023336 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 18:04:46.025846 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.025710 2562 manager.go:354] Recovery completed Jul 08 18:04:46.031362 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.031349 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.033846 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.033822 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.033918 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.033859 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.033918 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.033869 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.034391 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.034373 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:04:46.034391 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.034388 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:04:46.034391 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.034391 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.034561 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.034405 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:04:46.035051 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.035037 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.035109 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.035063 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.035109 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.035072 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.036351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.036340 2562 policy_none.go:49] "None policy: Start" Jul 08 18:04:46.036398 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.036355 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:04:46.036398 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.036365 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:04:46.038210 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.038198 2562 policy_none.go:47] "Start" Jul 08 18:04:46.042088 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.042012 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9eb5ef0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.033846 +0000 UTC m=+0.373289998,LastTimestamp:2026-07-08 18:04:46.033846 +0000 UTC m=+0.373289998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.050772 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.050710 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9eba575 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.033864053 +0000 UTC m=+0.373308052,LastTimestamp:2026-07-08 18:04:46.033864053 +0000 UTC m=+0.373308052,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.059872 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.059816 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9ebca0c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.03387342 +0000 UTC m=+0.373317419,LastTimestamp:2026-07-08 18:04:46.03387342 +0000 UTC m=+0.373317419,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.071647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.071635 2562 manager.go:337] "Starting Device Plugin manager" Jul 08 18:04:46.071706 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.071672 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:04:46.071706 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.071682 2562 server.go:87] "Starting device plugin registration server" Jul 08 18:04:46.071913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.071902 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:04:46.071965 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.071917 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:04:46.072013 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.072000 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:04:46.072091 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.072078 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:04:46.072156 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.072093 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:04:46.072519 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.072427 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-126.ec2.internal.18c0629ea9eb5ef0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9eb5ef0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.033846 +0000 UTC m=+0.373289998,LastTimestamp:2026-07-08 18:04:46.035051238 +0000 UTC m=+0.374495236,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.072619 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.072535 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:04:46.072619 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.072585 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.082940 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.082877 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-126.ec2.internal.18c0629ea9eba575\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9eba575 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.033864053 +0000 UTC m=+0.373308052,LastTimestamp:2026-07-08 18:04:46.035067791 +0000 UTC m=+0.374511790,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.092679 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.092616 2562 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-126.ec2.internal.18c0629ea9ebca0c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629ea9ebca0c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-126.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.03387342 +0000 UTC m=+0.373317419,LastTimestamp:2026-07-08 18:04:46.035076243 +0000 UTC m=+0.374520242,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.103184 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.103116 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-126.ec2.internal.18c0629eacb86cd1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-126.ec2.internal,UID:ip-10-0-131-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-131-126.ec2.internal,},FirstTimestamp:2026-07-08 18:04:46.080838865 +0000 UTC m=+0.420282852,LastTimestamp:2026-07-08 18:04:46.080838865 +0000 UTC m=+0.420282852,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-126.ec2.internal,}" Jul 08 18:04:46.125865 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.125847 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsstq" Jul 08 18:04:46.132072 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.132055 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsstq" Jul 08 18:04:46.172369 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.172348 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.173248 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.173232 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.173325 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.173262 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.173325 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.173273 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.173325 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.173304 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.182221 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.182203 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:04:46.182304 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.182245 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:04:46.182304 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.182270 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:04:46.182375 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.182306 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:04:46.184535 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.184521 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:46.190006 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.189959 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.190006 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.189981 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-126.ec2.internal\": node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.231779 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.231753 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.283027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.283000 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal"] Jul 08 18:04:46.283142 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.283074 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.284088 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.284073 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.284150 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.284099 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.284150 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.284109 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.287084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287071 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.287218 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287197 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.287275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287236 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.287788 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287771 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.287871 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287798 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.287871 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287810 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.287871 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287779 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.287871 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287866 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.288053 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.287875 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.289880 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.289865 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.289965 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.289887 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:46.290502 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.290488 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:46.290572 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.290519 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:46.290572 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.290533 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:46.313665 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.313645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.313748 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.313669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.313748 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.313686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9496e6239949feac3069ae863246fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-131-126.ec2.internal\" (UID: \"a9496e6239949feac3069ae863246fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.316983 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.316966 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-126.ec2.internal\" not found" node="ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.321236 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.321222 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-126.ec2.internal\" not found" node="ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.332661 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.332631 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.414272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.414272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.414451 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414300 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9496e6239949feac3069ae863246fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-131-126.ec2.internal\" (UID: \"a9496e6239949feac3069ae863246fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.414451 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414332 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9496e6239949feac3069ae863246fd9-config\") pod \"kube-apiserver-proxy-ip-10-0-131-126.ec2.internal\" (UID: \"a9496e6239949feac3069ae863246fd9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.414451 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.414451 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.414335 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74d2401f41dfe9ac83ad0bbfb5d0771e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal\" (UID: \"74d2401f41dfe9ac83ad0bbfb5d0771e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.433398 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.433373 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.534176 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.534095 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.620668 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.620645 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.624027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.624005 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:46.634467 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.634440 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.734966 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.734935 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.835487 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.835417 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.935881 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:46.935855 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:46.955027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.955006 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:04:46.955506 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:46.955116 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:04:47.036019 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:47.035993 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:47.135316 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.135234 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:59:46 +0000 UTC" deadline="2028-04-15 12:50:26.947694576 +0000 UTC" Jul 08 18:04:47.135316 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.135268 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15522h45m39.8124298s" Jul 08 18:04:47.136415 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:47.136398 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:47.163522 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.163493 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:47.183786 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.183751 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:04:47.184810 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.184794 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:04:47.184897 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.184825 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:04:47.184897 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.184849 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:04:47.237033 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:47.237010 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:47.272878 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.272854 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:47.338506 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:47.338481 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-126.ec2.internal\" not found" Jul 08 18:04:47.392283 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.392227 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:47.412692 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.412661 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" Jul 08 18:04:47.423705 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.423685 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:04:47.424639 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.424627 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" Jul 08 18:04:47.436666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.436649 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:04:47.795639 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:47.795593 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9496e6239949feac3069ae863246fd9.slice/crio-7a3fb231d9f6d328441a1137fce341fa60f83b30ca27cc5a2e6bc2f7f102339d WatchSource:0}: Error finding container 7a3fb231d9f6d328441a1137fce341fa60f83b30ca27cc5a2e6bc2f7f102339d: Status 404 returned error can't find the container with id 7a3fb231d9f6d328441a1137fce341fa60f83b30ca27cc5a2e6bc2f7f102339d Jul 08 18:04:47.800731 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.800713 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:04:47.865494 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:47.865437 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d2401f41dfe9ac83ad0bbfb5d0771e.slice/crio-94cd0ab11ce9b942d4e3be875e9e70933ea7017015dcbc0c8f128343ba845111 WatchSource:0}: Error finding container 94cd0ab11ce9b942d4e3be875e9e70933ea7017015dcbc0c8f128343ba845111: Status 404 returned error can't find the container with id 94cd0ab11ce9b942d4e3be875e9e70933ea7017015dcbc0c8f128343ba845111 Jul 08 18:04:47.996082 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.996009 2562 apiserver.go:52] "Watching apiserver" Jul 08 18:04:47.998714 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:47.998700 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:04:48.002684 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.002667 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:04:48.003027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.003009 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-8mzhb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb","openshift-cluster-node-tuning-operator/tuned-c2jdn","openshift-multus/multus-additional-cni-plugins-kplpd","openshift-multus/network-metrics-daemon-b6q6h","openshift-network-operator/iptables-alerter-m7h4c","openshift-ovn-kubernetes/ovnkube-node-jkq98","kube-system/global-pull-secret-syncer-2tv7h","kube-system/konnectivity-agent-d5slx","kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal","openshift-dns/node-resolver-wgwr6","openshift-image-registry/node-ca-44g8z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal","openshift-multus/multus-5dhq5"] Jul 08 18:04:48.005919 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.005904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.008112 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.008092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.008919 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.008899 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:04:48.009031 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.009095 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009037 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-522dx\"" Jul 08 18:04:48.009095 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009076 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:04:48.009196 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009102 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:04:48.009196 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.009196 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.009148 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:04:48.010389 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.010373 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.010473 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.010408 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.010549 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.010454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.010613 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.010550 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bxd82\"" Jul 08 18:04:48.012675 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.012654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fsr2l\"" Jul 08 18:04:48.012675 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.012668 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.012872 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.012855 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.013170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.013071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.013690 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.013674 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:04:48.016812 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.016790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.016892 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.016872 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:48.018847 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.018827 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.019113 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.019244 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019227 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:04:48.019570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019553 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.019633 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019566 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:04:48.019725 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:04:48.019831 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.019817 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ggjgj\"" Jul 08 18:04:48.021526 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.021698 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-bin\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.021770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.021770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-var-lib-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.021770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021762 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-netd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.021867 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4k9t\" (UniqueName: \"kubernetes.io/projected/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-kube-api-access-r4k9t\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.021867 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-kubernetes\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.021933 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-sys\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.021933 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.021933 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46pst\" (UniqueName: \"kubernetes.io/projected/9bb5a43f-a35e-44df-a17b-e910cdcbb798-kube-api-access-46pst\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.021933 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-netns\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-conf\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.021995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-tmp\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-slash\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-hosts-file\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-tmp-dir\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022098 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-host\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-os-release\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022156 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-systemd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-modprobe-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-systemd\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f79pp\" (UniqueName: \"kubernetes.io/projected/bd748a79-b017-4b08-8450-5d0734cefe9e-kube-api-access-f79pp\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysconfig\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-run\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-var-lib-kubelet\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-system-cni-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-log-socket\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-systemd-units\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022461 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-config\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-env-overrides\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovn-node-metrics-cert\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-tuned\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-etc-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-script-lib\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-lib-modules\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl4s4\" (UniqueName: \"kubernetes.io/projected/3d94a458-8e18-44f8-a77e-15d1b1b92654-kube-api-access-zl4s4\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-cnibin\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-kubelet\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-ovn\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.022835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.022843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-node-log\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.023378 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023331 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.023378 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023360 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.023488 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nhwml\"" Jul 08 18:04:48.023557 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023491 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:04:48.023859 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023841 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.023934 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023871 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:04:48.023934 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023927 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:04:48.024037 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.023952 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-42pt6\"" Jul 08 18:04:48.026185 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.026169 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:04:48.026328 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.026315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.026794 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.026777 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9cst8\"" Jul 08 18:04:48.028829 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.028696 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.028829 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.028737 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:04:48.028931 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.028833 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.028931 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.028896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7w4fd\"" Jul 08 18:04:48.029042 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.029026 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.029119 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.029099 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:48.031451 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.031437 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:48.031562 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.031532 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:48.031666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.031631 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:04:48.033969 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.033952 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.036595 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.036576 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:04:48.036752 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.036589 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:04:48.036752 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.036622 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:04:48.036752 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.036655 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s7lt7\"" Jul 08 18:04:48.056583 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.056567 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lwxtm" Jul 08 18:04:48.065657 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.065639 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lwxtm" Jul 08 18:04:48.113150 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.113130 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:04:48.123746 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-sys-fs\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.123815 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-multus\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.123815 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.123815 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-netns\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.123815 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123803 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-tmp\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.123999 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.123999 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-netns\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.123999 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87sm9\" (UniqueName: \"kubernetes.io/projected/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-kube-api-access-87sm9\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.123999 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.123969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-os-release\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-host\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-os-release\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-netns\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-multus-certs\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124081 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvskx\" (UniqueName: \"kubernetes.io/projected/96b06878-7ebd-47c8-9232-2afab2afc896-kube-api-access-lvskx\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-modprobe-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysconfig\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124130 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:04:48.124170 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124146 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-host\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-run\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysconfig\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124218 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-modprobe-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124229 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-run\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-dbus\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-log-socket\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124304 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-systemd-units\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124357 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-systemd-units\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124360 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-log-socket\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-config\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-env-overrides\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124413 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-tuned\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-cni-binary-copy\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.124642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-script-lib\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-lib-modules\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl4s4\" (UniqueName: \"kubernetes.io/projected/3d94a458-8e18-44f8-a77e-15d1b1b92654-kube-api-access-zl4s4\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-registration-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9487t\" (UniqueName: \"kubernetes.io/projected/19d67e3b-9d7e-4089-ac36-2559bf0caf72-kube-api-access-9487t\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-kubelet\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-kubelet\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-var-lib-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-netd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4k9t\" (UniqueName: \"kubernetes.io/projected/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-kube-api-access-r4k9t\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-kubernetes\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-sys\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124878 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.125482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124945 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-env-overrides\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-socket-dir-parent\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.124993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c87a352-a206-4468-bda7-9cecf2f87adb-agent-certs\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-var-lib-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125026 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-config\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125024 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46pst\" (UniqueName: \"kubernetes.io/projected/9bb5a43f-a35e-44df-a17b-e910cdcbb798-kube-api-access-46pst\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125113 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovnkube-script-lib\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-os-release\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-conf\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125266 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-d\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125271 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-kubelet\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-conf-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8z8j\" (UniqueName: \"kubernetes.io/projected/506624db-3fe5-4cec-b36c-945e7006b9ed-kube-api-access-c8z8j\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125411 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-netd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125443 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-lib-modules\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.126361 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-sys\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125541 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-kubernetes\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125445 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-sysctl-conf\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-slash\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-hosts-file\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-tmp-dir\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-slash\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-hosts-file\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa3db00-98df-4643-bef0-a42e0d51140e-iptables-alerter-script\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-system-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c87a352-a206-4468-bda7-9cecf2f87adb-konnectivity-ca\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-systemd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125789 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-systemd\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f79pp\" (UniqueName: \"kubernetes.io/projected/bd748a79-b017-4b08-8450-5d0734cefe9e-kube-api-access-f79pp\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125815 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-systemd\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.127064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-cnibin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125897 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-systemd\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-var-lib-kubelet\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-system-cni-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-tmp-dir\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.125989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-bin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-multus-daemon-config\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19d67e3b-9d7e-4089-ac36-2559bf0caf72-host\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-system-cni-dir\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovn-node-metrics-cert\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126117 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd748a79-b017-4b08-8450-5d0734cefe9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126135 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d94a458-8e18-44f8-a77e-15d1b1b92654-var-lib-kubelet\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126138 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa3db00-98df-4643-bef0-a42e0d51140e-host-slash\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126168 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126192 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-hostroot\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.127620 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-socket-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-kubelet-config\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19d67e3b-9d7e-4089-ac36-2559bf0caf72-serviceca\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-etc-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-cnibin\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126348 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-etc-openvswitch\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-run-ovn-kubernetes\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4rrb\" (UniqueName: \"kubernetes.io/projected/bfa3db00-98df-4643-bef0-a42e0d51140e-kube-api-access-t4rrb\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126507 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-k8s-cni-cncf-io\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-device-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-ovn\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd748a79-b017-4b08-8450-5d0734cefe9e-cnibin\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-node-log\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-bin\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126707 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-node-log\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-run-ovn\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-etc-kubernetes\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.128647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.126751 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9bb5a43f-a35e-44df-a17b-e910cdcbb798-host-cni-bin\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.128647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.127797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-etc-tuned\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.128647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.127806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d94a458-8e18-44f8-a77e-15d1b1b92654-tmp\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.128647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.128370 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bb5a43f-a35e-44df-a17b-e910cdcbb798-ovn-node-metrics-cert\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.133692 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.133668 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl4s4\" (UniqueName: \"kubernetes.io/projected/3d94a458-8e18-44f8-a77e-15d1b1b92654-kube-api-access-zl4s4\") pod \"tuned-c2jdn\" (UID: \"3d94a458-8e18-44f8-a77e-15d1b1b92654\") " pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.133793 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.133746 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46pst\" (UniqueName: \"kubernetes.io/projected/9bb5a43f-a35e-44df-a17b-e910cdcbb798-kube-api-access-46pst\") pod \"ovnkube-node-jkq98\" (UID: \"9bb5a43f-a35e-44df-a17b-e910cdcbb798\") " pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.134611 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.134588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f79pp\" (UniqueName: \"kubernetes.io/projected/bd748a79-b017-4b08-8450-5d0734cefe9e-kube-api-access-f79pp\") pod \"multus-additional-cni-plugins-kplpd\" (UID: \"bd748a79-b017-4b08-8450-5d0734cefe9e\") " pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.135152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.135136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4k9t\" (UniqueName: \"kubernetes.io/projected/472a4a0a-88f5-4a4b-b095-8cb740e75cfa-kube-api-access-r4k9t\") pod \"node-resolver-wgwr6\" (UID: \"472a4a0a-88f5-4a4b-b095-8cb740e75cfa\") " pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.186895 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.186842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" event={"ID":"74d2401f41dfe9ac83ad0bbfb5d0771e","Type":"ContainerStarted","Data":"94cd0ab11ce9b942d4e3be875e9e70933ea7017015dcbc0c8f128343ba845111"} Jul 08 18:04:48.188474 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.188432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" event={"ID":"a9496e6239949feac3069ae863246fd9","Type":"ContainerStarted","Data":"7a3fb231d9f6d328441a1137fce341fa60f83b30ca27cc5a2e6bc2f7f102339d"} Jul 08 18:04:48.227643 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-socket-dir-parent\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227643 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c87a352-a206-4468-bda7-9cecf2f87adb-agent-certs\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-conf-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8z8j\" (UniqueName: \"kubernetes.io/projected/506624db-3fe5-4cec-b36c-945e7006b9ed-kube-api-access-c8z8j\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa3db00-98df-4643-bef0-a42e0d51140e-iptables-alerter-script\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-system-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227743 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-socket-dir-parent\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c87a352-a206-4468-bda7-9cecf2f87adb-konnectivity-ca\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-conf-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-cnibin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227842 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-system-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.227877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-bin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-multus-daemon-config\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227907 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-cnibin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19d67e3b-9d7e-4089-ac36-2559bf0caf72-host\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227962 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-bin\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.227976 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.227984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa3db00-98df-4643-bef0-a42e0d51140e-host-slash\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228012 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228052 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19d67e3b-9d7e-4089-ac36-2559bf0caf72-host\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa3db00-98df-4643-bef0-a42e0d51140e-host-slash\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-multus-cni-dir\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.228127 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:48.728057287 +0000 UTC m=+3.067501295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-hostroot\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-socket-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-kubelet-config\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19d67e3b-9d7e-4089-ac36-2559bf0caf72-serviceca\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.228442 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228278 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-hostroot\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.228370 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228372 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-kubelet-config\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.228413 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:48.728398248 +0000 UTC m=+3.067842237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228424 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-socket-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa3db00-98df-4643-bef0-a42e0d51140e-iptables-alerter-script\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c87a352-a206-4468-bda7-9cecf2f87adb-konnectivity-ca\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4rrb\" (UniqueName: \"kubernetes.io/projected/bfa3db00-98df-4643-bef0-a42e0d51140e-kube-api-access-t4rrb\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-k8s-cni-cncf-io\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-device-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-multus-daemon-config\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-k8s-cni-cncf-io\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-etc-kubernetes\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228674 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-device-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-sys-fs\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228721 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-etc-kubernetes\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.229359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-multus\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228764 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-sys-fs\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228801 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-cni-multus\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87sm9\" (UniqueName: \"kubernetes.io/projected/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-kube-api-access-87sm9\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19d67e3b-9d7e-4089-ac36-2559bf0caf72-serviceca\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-os-release\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-netns\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228886 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-os-release\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-multus-certs\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228918 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-netns\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvskx\" (UniqueName: \"kubernetes.io/projected/96b06878-7ebd-47c8-9232-2afab2afc896-kube-api-access-lvskx\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-run-multus-certs\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.228966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-dbus\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-cni-binary-copy\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-registration-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229092 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d6085b44-3dca-430b-a48d-0b8c9dcc8942-dbus\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.230026 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9487t\" (UniqueName: \"kubernetes.io/projected/19d67e3b-9d7e-4089-ac36-2559bf0caf72-kube-api-access-9487t\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-kubelet\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229171 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/506624db-3fe5-4cec-b36c-945e7006b9ed-registration-dir\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229231 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96b06878-7ebd-47c8-9232-2afab2afc896-host-var-lib-kubelet\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.229635 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96b06878-7ebd-47c8-9232-2afab2afc896-cni-binary-copy\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.230588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.230500 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c87a352-a206-4468-bda7-9cecf2f87adb-agent-certs\") pod \"konnectivity-agent-d5slx\" (UID: \"0c87a352-a206-4468-bda7-9cecf2f87adb\") " pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.245197 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.245173 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:48.245306 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.245203 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:48.245306 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.245219 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:48.245306 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.245294 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:48.745277021 +0000 UTC m=+3.084721033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:48.246514 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.246454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4rrb\" (UniqueName: \"kubernetes.io/projected/bfa3db00-98df-4643-bef0-a42e0d51140e-kube-api-access-t4rrb\") pod \"iptables-alerter-m7h4c\" (UID: \"bfa3db00-98df-4643-bef0-a42e0d51140e\") " pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.247014 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.246994 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8z8j\" (UniqueName: \"kubernetes.io/projected/506624db-3fe5-4cec-b36c-945e7006b9ed-kube-api-access-c8z8j\") pod \"aws-ebs-csi-driver-node-9shxb\" (UID: \"506624db-3fe5-4cec-b36c-945e7006b9ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.247014 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.247009 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvskx\" (UniqueName: \"kubernetes.io/projected/96b06878-7ebd-47c8-9232-2afab2afc896-kube-api-access-lvskx\") pod \"multus-5dhq5\" (UID: \"96b06878-7ebd-47c8-9232-2afab2afc896\") " pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.247759 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.247737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9487t\" (UniqueName: \"kubernetes.io/projected/19d67e3b-9d7e-4089-ac36-2559bf0caf72-kube-api-access-9487t\") pod \"node-ca-44g8z\" (UID: \"19d67e3b-9d7e-4089-ac36-2559bf0caf72\") " pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.247853 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.247806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87sm9\" (UniqueName: \"kubernetes.io/projected/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-kube-api-access-87sm9\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.318938 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.318916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:04:48.326049 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.326035 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wgwr6" Jul 08 18:04:48.330205 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.330187 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb5a43f_a35e_44df_a17b_e910cdcbb798.slice/crio-945cffcc6ed75ee6637be314c7a654c266244ecc998e395afe677e8ebfd3c233 WatchSource:0}: Error finding container 945cffcc6ed75ee6637be314c7a654c266244ecc998e395afe677e8ebfd3c233: Status 404 returned error can't find the container with id 945cffcc6ed75ee6637be314c7a654c266244ecc998e395afe677e8ebfd3c233 Jul 08 18:04:48.332953 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.332935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" Jul 08 18:04:48.337359 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.336176 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472a4a0a_88f5_4a4b_b095_8cb740e75cfa.slice/crio-acf27117d75f853a2c8be6c2999af870f57a03224577e1a4759c8e67800b5b06 WatchSource:0}: Error finding container acf27117d75f853a2c8be6c2999af870f57a03224577e1a4759c8e67800b5b06: Status 404 returned error can't find the container with id acf27117d75f853a2c8be6c2999af870f57a03224577e1a4759c8e67800b5b06 Jul 08 18:04:48.338655 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.338567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kplpd" Jul 08 18:04:48.343495 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.343470 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d94a458_8e18_44f8_a77e_15d1b1b92654.slice/crio-dd4a320a6021216769ef3697840644470279bb8a461698aa92123d9fb8bedcac WatchSource:0}: Error finding container dd4a320a6021216769ef3697840644470279bb8a461698aa92123d9fb8bedcac: Status 404 returned error can't find the container with id dd4a320a6021216769ef3697840644470279bb8a461698aa92123d9fb8bedcac Jul 08 18:04:48.344955 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.344933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m7h4c" Jul 08 18:04:48.347552 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.347533 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd748a79_b017_4b08_8450_5d0734cefe9e.slice/crio-b47f0e1913299306a05c1be944a0dd8f5bbfaef217d3fd010fc973611ed492a3 WatchSource:0}: Error finding container b47f0e1913299306a05c1be944a0dd8f5bbfaef217d3fd010fc973611ed492a3: Status 404 returned error can't find the container with id b47f0e1913299306a05c1be944a0dd8f5bbfaef217d3fd010fc973611ed492a3 Jul 08 18:04:48.351100 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.350981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:04:48.354444 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.354422 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa3db00_98df_4643_bef0_a42e0d51140e.slice/crio-95915f4262eb74889433b6967b98451f2c8e131259fae28c771eb744f8fac5cb WatchSource:0}: Error finding container 95915f4262eb74889433b6967b98451f2c8e131259fae28c771eb744f8fac5cb: Status 404 returned error can't find the container with id 95915f4262eb74889433b6967b98451f2c8e131259fae28c771eb744f8fac5cb Jul 08 18:04:48.358011 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.357995 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5dhq5" Jul 08 18:04:48.359581 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.359563 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c87a352_a206_4468_bda7_9cecf2f87adb.slice/crio-fc3e78102444b0ccb83f3754836b5b4db1655334674c9a7d3e7cdeeeb028f66a WatchSource:0}: Error finding container fc3e78102444b0ccb83f3754836b5b4db1655334674c9a7d3e7cdeeeb028f66a: Status 404 returned error can't find the container with id fc3e78102444b0ccb83f3754836b5b4db1655334674c9a7d3e7cdeeeb028f66a Jul 08 18:04:48.364769 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.364754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" Jul 08 18:04:48.367317 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.367295 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b06878_7ebd_47c8_9232_2afab2afc896.slice/crio-434afbfa7c6c8b7c6f529f4ed557155d092b6e139389ec076b42e1a993622646 WatchSource:0}: Error finding container 434afbfa7c6c8b7c6f529f4ed557155d092b6e139389ec076b42e1a993622646: Status 404 returned error can't find the container with id 434afbfa7c6c8b7c6f529f4ed557155d092b6e139389ec076b42e1a993622646 Jul 08 18:04:48.368980 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.368962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-44g8z" Jul 08 18:04:48.374748 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.374729 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod506624db_3fe5_4cec_b36c_945e7006b9ed.slice/crio-988a5552031892d0444371d708250f9a922d37387419d4988cf59cfd0e3282e9 WatchSource:0}: Error finding container 988a5552031892d0444371d708250f9a922d37387419d4988cf59cfd0e3282e9: Status 404 returned error can't find the container with id 988a5552031892d0444371d708250f9a922d37387419d4988cf59cfd0e3282e9 Jul 08 18:04:48.379266 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:04:48.379249 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d67e3b_9d7e_4089_ac36_2559bf0caf72.slice/crio-fb3e02f3f132579bdb462cba8cd65aaefae98484772571b8f43e78373216653b WatchSource:0}: Error finding container fb3e02f3f132579bdb462cba8cd65aaefae98484772571b8f43e78373216653b: Status 404 returned error can't find the container with id fb3e02f3f132579bdb462cba8cd65aaefae98484772571b8f43e78373216653b Jul 08 18:04:48.733679 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.733585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:48.733839 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.733710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:48.733839 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.733754 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:48.733839 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.733824 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:48.733839 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.733835 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.733813839 +0000 UTC m=+4.073257832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:48.734025 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.733880 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.733863041 +0000 UTC m=+4.073307029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:48.834505 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:48.834384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:48.834648 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.834550 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:48.834648 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.834576 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:48.834648 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.834589 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:48.834774 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:48.834653 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:49.834630566 +0000 UTC m=+4.174074575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:49.068682 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.068513 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:59:48 +0000 UTC" deadline="2028-01-02 21:30:16.972942316 +0000 UTC" Jul 08 18:04:49.068682 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.068545 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13035h25m27.904402074s" Jul 08 18:04:49.182518 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.182488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:49.182670 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.182622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:49.192897 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.192834 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-44g8z" event={"ID":"19d67e3b-9d7e-4089-ac36-2559bf0caf72","Type":"ContainerStarted","Data":"fb3e02f3f132579bdb462cba8cd65aaefae98484772571b8f43e78373216653b"} Jul 08 18:04:49.196593 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.196568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d5slx" event={"ID":"0c87a352-a206-4468-bda7-9cecf2f87adb","Type":"ContainerStarted","Data":"fc3e78102444b0ccb83f3754836b5b4db1655334674c9a7d3e7cdeeeb028f66a"} Jul 08 18:04:49.205799 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.205774 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" event={"ID":"3d94a458-8e18-44f8-a77e-15d1b1b92654","Type":"ContainerStarted","Data":"dd4a320a6021216769ef3697840644470279bb8a461698aa92123d9fb8bedcac"} Jul 08 18:04:49.207900 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.207874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgwr6" event={"ID":"472a4a0a-88f5-4a4b-b095-8cb740e75cfa","Type":"ContainerStarted","Data":"acf27117d75f853a2c8be6c2999af870f57a03224577e1a4759c8e67800b5b06"} Jul 08 18:04:49.210072 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.210025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"945cffcc6ed75ee6637be314c7a654c266244ecc998e395afe677e8ebfd3c233"} Jul 08 18:04:49.212616 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.212580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" event={"ID":"506624db-3fe5-4cec-b36c-945e7006b9ed","Type":"ContainerStarted","Data":"988a5552031892d0444371d708250f9a922d37387419d4988cf59cfd0e3282e9"} Jul 08 18:04:49.217343 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.217290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dhq5" event={"ID":"96b06878-7ebd-47c8-9232-2afab2afc896","Type":"ContainerStarted","Data":"434afbfa7c6c8b7c6f529f4ed557155d092b6e139389ec076b42e1a993622646"} Jul 08 18:04:49.219566 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.219540 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m7h4c" event={"ID":"bfa3db00-98df-4643-bef0-a42e0d51140e","Type":"ContainerStarted","Data":"95915f4262eb74889433b6967b98451f2c8e131259fae28c771eb744f8fac5cb"} Jul 08 18:04:49.225549 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.225529 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerStarted","Data":"b47f0e1913299306a05c1be944a0dd8f5bbfaef217d3fd010fc973611ed492a3"} Jul 08 18:04:49.742727 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.742681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:49.742896 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.742741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:49.742896 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.742824 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:49.742896 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.742853 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:49.742896 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.742894 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:51.742874655 +0000 UTC m=+6.082318647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:49.743124 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.742914 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:51.742904145 +0000 UTC m=+6.082348131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:49.843075 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:49.843032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:49.843256 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.843240 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:49.843340 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.843265 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:49.843340 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.843278 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:49.843340 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:49.843330 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:51.843312596 +0000 UTC m=+6.182756596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:50.069132 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:50.069024 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:59:48 +0000 UTC" deadline="2027-12-25 16:43:03.465124323 +0000 UTC" Jul 08 18:04:50.069132 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:50.069084 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12838h38m13.396044349s" Jul 08 18:04:50.185325 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:50.185301 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:50.185485 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:50.185421 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:50.185835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:50.185810 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:50.185953 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:50.185903 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:51.182676 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:51.182640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:51.183111 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.182789 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:51.760930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:51.760985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.761147 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.761218 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:55.761190735 +0000 UTC m=+10.100634735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.761611 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:51.761732 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.761658 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:55.761643316 +0000 UTC m=+10.101087305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:51.861744 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:51.861715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:51.861903 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.861884 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:51.861903 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.861901 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:51.861994 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.861913 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:51.861994 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:51.861962 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:04:55.861945258 +0000 UTC m=+10.201389245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:52.183964 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:52.183426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:52.183964 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:52.183428 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:52.183964 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:52.183570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:52.183964 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:52.183677 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:53.182979 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:53.182946 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:53.183162 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:53.183087 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:54.185149 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:54.185120 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:54.185603 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:54.185217 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:54.185666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:54.185623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:54.185740 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:54.185712 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:55.182808 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:55.182770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:55.182992 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.182908 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:55.792590 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:55.792548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:55.793038 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:55.792611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:55.793038 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.792776 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:55.793038 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.792837 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:03.792817805 +0000 UTC m=+18.132261792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:04:55.793247 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.793230 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:55.793317 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.793286 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:03.793270173 +0000 UTC m=+18.132714160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:04:55.893181 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:55.893133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:55.893383 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.893363 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:04:55.893432 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.893389 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:04:55.893432 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.893398 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:55.893541 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:55.893466 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:03.893433521 +0000 UTC m=+18.232877520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:04:56.183636 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:56.183558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:56.183796 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:56.183668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:56.184198 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:56.184048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:56.184198 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:56.184140 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:57.183191 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:57.182979 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:57.183681 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:57.183304 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:04:58.183306 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:58.183266 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:04:58.183873 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:58.183279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:04:58.183873 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:58.183487 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:04:58.183873 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:58.183556 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:04:59.183441 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:04:59.183409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:04:59.183858 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:04:59.183562 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:00.182948 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:00.182922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:00.183094 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:00.182922 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:00.183094 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:00.183037 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:00.183211 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:00.183100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:01.183499 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:01.183465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:01.183895 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:01.183590 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:02.182699 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:02.182660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:02.182875 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:02.182664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:02.182875 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:02.182836 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:02.183025 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:02.182958 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:03.182708 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:03.182676 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:03.183140 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.182815 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:03.855769 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:03.855733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:03.856000 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:03.855784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:03.856000 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.855920 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:05:03.856000 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.855918 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:05:03.856000 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.855977 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:19.855957169 +0000 UTC m=+34.195401161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:05:03.856000 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.855992 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:19.855986075 +0000 UTC m=+34.195430062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:05:03.956738 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:03.956698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:03.956963 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.956894 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:05:03.956963 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.956914 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:05:03.956963 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.956926 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:05:03.957133 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:03.956989 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:19.956971212 +0000 UTC m=+34.296415199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:05:04.183143 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:04.183106 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:04.183539 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:04.183145 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:04.183539 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:04.183236 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:04.183539 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:04.183375 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:05.183396 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:05.183363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:05.183803 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:05.183489 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:06.183613 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:06.183581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:06.184068 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:06.183681 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:06.184068 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:06.183749 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:06.184068 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:06.183822 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:07.182738 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:07.182710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:07.182917 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:07.182831 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:08.183101 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:08.183063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:08.183101 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:08.183106 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:08.183613 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:08.183187 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:08.183613 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:08.183548 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:09.183519 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.183307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:09.184185 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:09.183566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:09.281081 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.281046 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" event={"ID":"3d94a458-8e18-44f8-a77e-15d1b1b92654","Type":"ContainerStarted","Data":"3e4b7fcfaaff06d60613ea96a1bddef28729134f341c9decfed656a508c45531"} Jul 08 18:05:09.282612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.282582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wgwr6" event={"ID":"472a4a0a-88f5-4a4b-b095-8cb740e75cfa","Type":"ContainerStarted","Data":"83dd42aecaae0209871b311ccf247fe910058e47f1da1be002261f90b4e4b734"} Jul 08 18:05:09.285699 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285678 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"ca0042cf59a87351d2bba5c7003585e1ecc8499dc354849ac135b337d181e65f"} Jul 08 18:05:09.285782 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"f2d4f746229fd9664ac686d25ad2d2c9ac2ce91a44413699738755eeeb007ac6"} Jul 08 18:05:09.285782 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"03fcae3c89ab1e3ab4533d6c0d5343537d5b42ef8584f30245bc2a8fb108463e"} Jul 08 18:05:09.285864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"bbed289eea8f54dd9244fa6dbcfc5004d3093db82498ec2e77f8048bdb80f3f5"} Jul 08 18:05:09.285864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"44734f61978e56f3b41c7b22842ca552c3d56ac3de4b0958f49dd1be481e6fe7"} Jul 08 18:05:09.285957 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.285871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"8d7608620225f3f211b6d276000db8b9fa3381b93558ba30307ddbb1b6c6e509"} Jul 08 18:05:09.286958 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.286937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" event={"ID":"506624db-3fe5-4cec-b36c-945e7006b9ed","Type":"ContainerStarted","Data":"553b8a2cdcbba9d6313956114ae60c1e00fdf2234caa868fd5ac2215d3c85faf"} Jul 08 18:05:09.288262 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.288204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5dhq5" event={"ID":"96b06878-7ebd-47c8-9232-2afab2afc896","Type":"ContainerStarted","Data":"927ec064a875733daf0bd988c0160c49721aa6623a27dda3ce34cda85ef51234"} Jul 08 18:05:09.289739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.289717 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="5bb84324bd3d7a76d5e7fa4db00de4b8149c1f0d0c824d951e22264a49ff7c19" exitCode=0 Jul 08 18:05:09.289834 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.289779 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"5bb84324bd3d7a76d5e7fa4db00de4b8149c1f0d0c824d951e22264a49ff7c19"} Jul 08 18:05:09.293966 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.293940 2562 generic.go:356] "Generic (PLEG): container finished" podID="74d2401f41dfe9ac83ad0bbfb5d0771e" containerID="b563c50d2764848f26958ebe06ce825c40cec7855c89ab5b3453b21b80d9895d" exitCode=0 Jul 08 18:05:09.294061 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.294027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" event={"ID":"74d2401f41dfe9ac83ad0bbfb5d0771e","Type":"ContainerDied","Data":"b563c50d2764848f26958ebe06ce825c40cec7855c89ab5b3453b21b80d9895d"} Jul 08 18:05:09.295323 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.295283 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" event={"ID":"a9496e6239949feac3069ae863246fd9","Type":"ContainerStarted","Data":"fe0e45b5cd3303588b0ca3cf9bfe0af194f22cb4e86795d11c8ad3d7c400ac1d"} Jul 08 18:05:09.296915 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.296898 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-44g8z" event={"ID":"19d67e3b-9d7e-4089-ac36-2559bf0caf72","Type":"ContainerStarted","Data":"5d802ce8912401f5a6f8be7ec9ad0dfade8950066557b8dbbdcc88d9f1acf65b"} Jul 08 18:05:09.298253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.298233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d5slx" event={"ID":"0c87a352-a206-4468-bda7-9cecf2f87adb","Type":"ContainerStarted","Data":"a263db793c397b8b7de18df4a0e9230cb3968854b930e7c874a4dd3ef4943e36"} Jul 08 18:05:09.300061 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.300028 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c2jdn" podStartSLOduration=3.3597807140000002 podStartE2EDuration="23.300015103s" totalImagesPullingTime="19.940234389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.299481671 +0000 UTC m=+23.638925671" watchObservedRunningTime="2026-07-08 18:05:09.300015103 +0000 UTC m=+23.639459114" Jul 08 18:05:09.347850 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.347673 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-126.ec2.internal" podStartSLOduration=22.347655709 podStartE2EDuration="22.347655709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.327263377 +0000 UTC m=+23.666707384" watchObservedRunningTime="2026-07-08 18:05:09.347655709 +0000 UTC m=+23.687099833" Jul 08 18:05:09.384058 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.384020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d5slx" podStartSLOduration=3.461694443 podStartE2EDuration="23.384006785s" totalImagesPullingTime="19.922312342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.3612813 +0000 UTC m=+23.700725309" watchObservedRunningTime="2026-07-08 18:05:09.384006785 +0000 UTC m=+23.723450786" Jul 08 18:05:09.384205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.384187 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5dhq5" podStartSLOduration=3.41706733 podStartE2EDuration="23.384183164s" totalImagesPullingTime="19.967115834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.383985106 +0000 UTC m=+23.723429110" watchObservedRunningTime="2026-07-08 18:05:09.384183164 +0000 UTC m=+23.723627172" Jul 08 18:05:09.415570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.415529 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wgwr6" podStartSLOduration=3.468911054 podStartE2EDuration="23.415511373s" totalImagesPullingTime="19.946600319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.400350788 +0000 UTC m=+23.739794795" watchObservedRunningTime="2026-07-08 18:05:09.415511373 +0000 UTC m=+23.754955435" Jul 08 18:05:09.415670 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:09.415611 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-44g8z" podStartSLOduration=11.562846163 podStartE2EDuration="23.415606002s" totalImagesPullingTime="11.852759839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:09.41531354 +0000 UTC m=+23.754757550" watchObservedRunningTime="2026-07-08 18:05:09.415606002 +0000 UTC m=+23.755050010" Jul 08 18:05:10.183300 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.183267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:10.183561 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:10.183373 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:10.183561 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.183400 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:10.183561 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:10.183475 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:10.302706 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.302675 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m7h4c" event={"ID":"bfa3db00-98df-4643-bef0-a42e0d51140e","Type":"ContainerStarted","Data":"88fcef0e41871c8b6f239cfaf2c38c1447560637029f7edb9c7f1fb5542342e1"} Jul 08 18:05:10.304636 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.304594 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" event={"ID":"74d2401f41dfe9ac83ad0bbfb5d0771e","Type":"ContainerStarted","Data":"3a2e29be8c85680a5a94c7c861dd8fdea94071a219aa6567d65ac61037bfa55f"} Jul 08 18:05:10.345937 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.345881 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m7h4c" podStartSLOduration=4.417393037 podStartE2EDuration="24.345865779s" totalImagesPullingTime="19.928472742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:10.323054466 +0000 UTC m=+24.662498483" watchObservedRunningTime="2026-07-08 18:05:10.345865779 +0000 UTC m=+24.685309787" Jul 08 18:05:10.346381 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:10.346344 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-126.ec2.internal" podStartSLOduration=23.346333893 podStartE2EDuration="23.346333893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:10.345802248 +0000 UTC m=+24.685246254" watchObservedRunningTime="2026-07-08 18:05:10.346333893 +0000 UTC m=+24.685777903" Jul 08 18:05:11.059201 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.059174 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:05:11.088550 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.088434 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:05:11.059198203Z","UUID":"c55d2e95-ce77-4cdc-9182-694d3330f84b","Handler":null,"Name":"","Endpoint":""} Jul 08 18:05:11.090024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.090006 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:05:11.090111 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.090047 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:05:11.182531 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.182466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:11.182662 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:11.182583 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:11.311713 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.311682 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"c8f7fa8822bb97b713eb35376fb7ab8170d9e0c7754ea838ecb285a7f4c10ee5"} Jul 08 18:05:11.313846 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:11.313814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" event={"ID":"506624db-3fe5-4cec-b36c-945e7006b9ed","Type":"ContainerStarted","Data":"8f1346d5b2f1943bb580f6554c94fe07287bb5c1d98401716891ba225781fba5"} Jul 08 18:05:12.182859 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:12.182827 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:12.183061 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:12.182827 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:12.183061 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:12.182956 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:12.183061 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:12.183023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:13.182974 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.182724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:13.182974 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:13.182959 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:13.322588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.322556 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" event={"ID":"9bb5a43f-a35e-44df-a17b-e910cdcbb798","Type":"ContainerStarted","Data":"2d6a0258d4eb62e42609e3d161c8d9dda1b26f90e0ce11e1131b0aa662051ec3"} Jul 08 18:05:13.323640 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.323623 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:13.323782 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.323692 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:13.323782 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.323780 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:13.326492 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.326448 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" event={"ID":"506624db-3fe5-4cec-b36c-945e7006b9ed","Type":"ContainerStarted","Data":"2fbd44c8d3c1bcaab875b26adf28f57c2e322f6f41fc0ba3e341f6cf0daeda2a"} Jul 08 18:05:13.342209 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.342189 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:13.342742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.342725 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:13.366752 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.365958 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" podStartSLOduration=7.3449776270000005 podStartE2EDuration="27.365940698s" totalImagesPullingTime="20.020963071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:13.357134714 +0000 UTC m=+27.696578717" watchObservedRunningTime="2026-07-08 18:05:13.365940698 +0000 UTC m=+27.705384707" Jul 08 18:05:13.377971 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:13.377927 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9shxb" podStartSLOduration=3.898653562 podStartE2EDuration="27.377909667s" totalImagesPullingTime="23.479256105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:13.377619582 +0000 UTC m=+27.717063590" watchObservedRunningTime="2026-07-08 18:05:13.377909667 +0000 UTC m=+27.717353676" Jul 08 18:05:14.183624 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.183539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:14.183624 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.183569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:14.184332 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:14.183678 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:14.184332 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:14.183823 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:14.265409 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.265366 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:05:14.265996 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.265966 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:05:14.330326 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.330128 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:05:14.330326 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:14.330257 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d5slx" Jul 08 18:05:15.182728 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:15.182698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:15.182913 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:15.182820 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:16.183431 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:16.183357 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:16.183887 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:16.183439 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:16.183887 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:16.183505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:16.183887 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:16.183551 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:16.336117 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:16.336080 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="58b48ad9d10becdb85eb80379f82f9558f9c72bfba05383b78854b3c2a30d18d" exitCode=0 Jul 08 18:05:16.336287 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:16.336124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"58b48ad9d10becdb85eb80379f82f9558f9c72bfba05383b78854b3c2a30d18d"} Jul 08 18:05:17.183181 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:17.183155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:17.183324 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:17.183249 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:18.183408 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:18.183381 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:18.183874 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:18.183389 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:18.183874 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:18.183497 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:18.183874 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:18.183559 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:18.342068 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:18.342036 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="751cf56285debd1dfd34c8dbc856ec5f5c3ce94a4da6b8d990f94c49eba7d468" exitCode=0 Jul 08 18:05:18.342226 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:18.342073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"751cf56285debd1dfd34c8dbc856ec5f5c3ce94a4da6b8d990f94c49eba7d468"} Jul 08 18:05:19.183562 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:19.183300 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:19.184009 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.183676 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:19.348955 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:19.348926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerStarted","Data":"60726d1232e8cdc075875910fa9394602f3789f19001d8495b642520a2fa21b5"} Jul 08 18:05:19.880613 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:19.880565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:19.880748 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:19.880623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:19.880748 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.880692 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:05:19.880833 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.880750 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:05:19.880833 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.880752 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret podName:d6085b44-3dca-430b-a48d-0b8c9dcc8942 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:51.880734975 +0000 UTC m=+66.220178964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret") pod "global-pull-secret-syncer-2tv7h" (UID: "d6085b44-3dca-430b-a48d-0b8c9dcc8942") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:05:19.880833 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.880821 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs podName:2b80a286-c8d8-4fe3-a6c7-103281e1dce7 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:51.880804311 +0000 UTC m=+66.220248299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs") pod "network-metrics-daemon-b6q6h" (UID: "2b80a286-c8d8-4fe3-a6c7-103281e1dce7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:05:19.981398 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:19.981368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:19.981564 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.981534 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:05:19.981564 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.981555 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:05:19.981638 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.981565 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tbv88 for pod openshift-network-diagnostics/network-check-target-8mzhb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:05:19.981638 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:19.981633 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88 podName:a1995353-4b17-4deb-a76c-6bba7b344da0 nodeName:}" failed. No retries permitted until 2026-07-08 18:05:51.981618849 +0000 UTC m=+66.321062838 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tbv88" (UniqueName: "kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88") pod "network-check-target-8mzhb" (UID: "a1995353-4b17-4deb-a76c-6bba7b344da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:05:20.183089 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.183022 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:20.183216 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:20.183118 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:20.183216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.183126 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:20.183296 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:20.183215 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:20.354973 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.354605 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="60726d1232e8cdc075875910fa9394602f3789f19001d8495b642520a2fa21b5" exitCode=0 Jul 08 18:05:20.354973 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.354656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"60726d1232e8cdc075875910fa9394602f3789f19001d8495b642520a2fa21b5"} Jul 08 18:05:20.572512 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.572471 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2tv7h"] Jul 08 18:05:20.572702 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.572651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:20.572962 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:20.572768 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:20.575234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.575203 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6q6h"] Jul 08 18:05:20.575340 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.575303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:20.575434 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:20.575413 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:20.576007 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.575985 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8mzhb"] Jul 08 18:05:20.576214 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:20.576194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:20.576313 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:20.576291 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:22.186646 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:22.186614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:22.187130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:22.186659 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:22.187130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:22.186734 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:22.187130 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:22.186743 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2tv7h" podUID="d6085b44-3dca-430b-a48d-0b8c9dcc8942" Jul 08 18:05:22.187130 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:22.186834 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6q6h" podUID="2b80a286-c8d8-4fe3-a6c7-103281e1dce7" Jul 08 18:05:22.187130 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:22.186896 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8mzhb" podUID="a1995353-4b17-4deb-a76c-6bba7b344da0" Jul 08 18:05:23.926869 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:23.926840 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-126.ec2.internal" event="NodeReady" Jul 08 18:05:23.927306 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:23.926998 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:05:24.014865 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.014833 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wxlzr"] Jul 08 18:05:24.031425 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.031392 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xgwh2"] Jul 08 18:05:24.031591 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.031511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.040229 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.040211 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:05:24.040315 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.040241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmswz\"" Jul 08 18:05:24.040356 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.040323 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:05:24.050130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.050073 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgwh2"] Jul 08 18:05:24.050130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.050120 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxlzr"] Jul 08 18:05:24.050269 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.050227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.057559 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.057537 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vlvs2"] Jul 08 18:05:24.064908 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.064875 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 18:05:24.065014 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.065002 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 18:05:24.065138 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.065125 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 18:05:24.065253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.065226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tfxt7\"" Jul 08 18:05:24.065423 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.065407 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 18:05:24.071426 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.071405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.074543 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.074522 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:05:24.075261 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.075244 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cd4hg\"" Jul 08 18:05:24.076051 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.076034 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:05:24.084825 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.084801 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vlvs2"] Jul 08 18:05:24.085357 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.085339 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:05:24.111420 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.111385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6039c0a5-514c-4508-8961-80a26b8ed4e6-tmp-dir\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.111590 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.111508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6039c0a5-514c-4508-8961-80a26b8ed4e6-config-volume\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.111590 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.111539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg5zt\" (UniqueName: \"kubernetes.io/projected/6039c0a5-514c-4508-8961-80a26b8ed4e6-kube-api-access-lg5zt\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.111671 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.111654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6039c0a5-514c-4508-8961-80a26b8ed4e6-metrics-tls\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.182738 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.182711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:24.182876 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.182711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:24.182930 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.182711 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:24.186002 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.185920 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:05:24.186129 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.186086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9s8r\"" Jul 08 18:05:24.186187 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.186166 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8pfq\"" Jul 08 18:05:24.186234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.186207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:05:24.186293 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.186278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:05:24.186355 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.186317 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:05:24.211905 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.211886 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6039c0a5-514c-4508-8961-80a26b8ed4e6-metrics-tls\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.211991 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.211913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32133831-f978-43c4-94fe-33bfe0490974-crio-socket\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.211991 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.211935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6039c0a5-514c-4508-8961-80a26b8ed4e6-tmp-dir\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.211991 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.211965 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32133831-f978-43c4-94fe-33bfe0490974-data-volume\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.211991 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.211989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw58w\" (UniqueName: \"kubernetes.io/projected/32133831-f978-43c4-94fe-33bfe0490974-kube-api-access-rw58w\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.212133 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32133831-f978-43c4-94fe-33bfe0490974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.212133 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6039c0a5-514c-4508-8961-80a26b8ed4e6-config-volume\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.212221 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg5zt\" (UniqueName: \"kubernetes.io/projected/6039c0a5-514c-4508-8961-80a26b8ed4e6-kube-api-access-lg5zt\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.212221 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6wz\" (UniqueName: \"kubernetes.io/projected/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-kube-api-access-tk6wz\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.212221 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32133831-f978-43c4-94fe-33bfe0490974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.212369 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212223 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6039c0a5-514c-4508-8961-80a26b8ed4e6-tmp-dir\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.212369 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-cert\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.212628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.212610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6039c0a5-514c-4508-8961-80a26b8ed4e6-config-volume\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.216333 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.216193 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6039c0a5-514c-4508-8961-80a26b8ed4e6-metrics-tls\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.223113 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.223092 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg5zt\" (UniqueName: \"kubernetes.io/projected/6039c0a5-514c-4508-8961-80a26b8ed4e6-kube-api-access-lg5zt\") pod \"dns-default-wxlzr\" (UID: \"6039c0a5-514c-4508-8961-80a26b8ed4e6\") " pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.313638 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6wz\" (UniqueName: \"kubernetes.io/projected/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-kube-api-access-tk6wz\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.313638 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32133831-f978-43c4-94fe-33bfe0490974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.313816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-cert\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.313816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32133831-f978-43c4-94fe-33bfe0490974-crio-socket\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.313816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32133831-f978-43c4-94fe-33bfe0490974-data-volume\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.313945 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.313819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw58w\" (UniqueName: \"kubernetes.io/projected/32133831-f978-43c4-94fe-33bfe0490974-kube-api-access-rw58w\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.314029 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.314004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32133831-f978-43c4-94fe-33bfe0490974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.314076 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.314031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32133831-f978-43c4-94fe-33bfe0490974-crio-socket\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.314204 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.314182 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32133831-f978-43c4-94fe-33bfe0490974-data-volume\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.314534 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.314450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32133831-f978-43c4-94fe-33bfe0490974-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.316060 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.316043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-cert\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.316151 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.316119 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32133831-f978-43c4-94fe-33bfe0490974-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.326173 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.326150 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6wz\" (UniqueName: \"kubernetes.io/projected/2c54a90d-3ccc-4291-a248-2a72dcfc71b2-kube-api-access-tk6wz\") pod \"ingress-canary-vlvs2\" (UID: \"2c54a90d-3ccc-4291-a248-2a72dcfc71b2\") " pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.328616 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.328583 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw58w\" (UniqueName: \"kubernetes.io/projected/32133831-f978-43c4-94fe-33bfe0490974-kube-api-access-rw58w\") pod \"insights-runtime-extractor-xgwh2\" (UID: \"32133831-f978-43c4-94fe-33bfe0490974\") " pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.340528 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.340509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:24.365760 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.365743 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xgwh2" Jul 08 18:05:24.382847 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.382823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vlvs2" Jul 08 18:05:24.554385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.554358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wxlzr"] Jul 08 18:05:24.558440 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:24.558297 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6039c0a5_514c_4508_8961_80a26b8ed4e6.slice/crio-7ff071e438932c059d78aa5a38a34d0915bbbe88305c2c8124f02b4ef798eb3c WatchSource:0}: Error finding container 7ff071e438932c059d78aa5a38a34d0915bbbe88305c2c8124f02b4ef798eb3c: Status 404 returned error can't find the container with id 7ff071e438932c059d78aa5a38a34d0915bbbe88305c2c8124f02b4ef798eb3c Jul 08 18:05:24.559505 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.559479 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vlvs2"] Jul 08 18:05:24.560012 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:24.559990 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c54a90d_3ccc_4291_a248_2a72dcfc71b2.slice/crio-bf48634b2faf22d976187054aa1be4bb7c25186ca59f9e0de174f083a15ff6dc WatchSource:0}: Error finding container bf48634b2faf22d976187054aa1be4bb7c25186ca59f9e0de174f083a15ff6dc: Status 404 returned error can't find the container with id bf48634b2faf22d976187054aa1be4bb7c25186ca59f9e0de174f083a15ff6dc Jul 08 18:05:24.560547 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:24.560480 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xgwh2"] Jul 08 18:05:24.560886 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:24.560844 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32133831_f978_43c4_94fe_33bfe0490974.slice/crio-54704b1cfbb632d2b87ccf79cc8712bca4f09d43c00c39e5dc70822bcd2ce32d WatchSource:0}: Error finding container 54704b1cfbb632d2b87ccf79cc8712bca4f09d43c00c39e5dc70822bcd2ce32d: Status 404 returned error can't find the container with id 54704b1cfbb632d2b87ccf79cc8712bca4f09d43c00c39e5dc70822bcd2ce32d Jul 08 18:05:25.373047 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:25.372968 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgwh2" event={"ID":"32133831-f978-43c4-94fe-33bfe0490974","Type":"ContainerStarted","Data":"54704b1cfbb632d2b87ccf79cc8712bca4f09d43c00c39e5dc70822bcd2ce32d"} Jul 08 18:05:25.375499 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:25.375413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vlvs2" event={"ID":"2c54a90d-3ccc-4291-a248-2a72dcfc71b2","Type":"ContainerStarted","Data":"bf48634b2faf22d976187054aa1be4bb7c25186ca59f9e0de174f083a15ff6dc"} Jul 08 18:05:25.377351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:25.377292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxlzr" event={"ID":"6039c0a5-514c-4508-8961-80a26b8ed4e6","Type":"ContainerStarted","Data":"7ff071e438932c059d78aa5a38a34d0915bbbe88305c2c8124f02b4ef798eb3c"} Jul 08 18:05:28.877783 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.877748 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-z529q"] Jul 08 18:05:28.909883 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.909854 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-z529q"] Jul 08 18:05:28.910032 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.909969 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:28.913435 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.913411 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 18:05:28.914356 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.914333 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 18:05:28.914493 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.914359 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 18:05:28.914493 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.914425 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 18:05:28.914684 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.914666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-48r4n\"" Jul 08 18:05:28.914792 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:28.914736 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 18:05:29.054216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.054180 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.054380 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.054238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.054380 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.054312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4dfb11-bdce-43ec-a938-730bac5de22d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.054505 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.054378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55nd8\" (UniqueName: \"kubernetes.io/projected/3a4dfb11-bdce-43ec-a938-730bac5de22d-kube-api-access-55nd8\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.155490 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.155396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55nd8\" (UniqueName: \"kubernetes.io/projected/3a4dfb11-bdce-43ec-a938-730bac5de22d-kube-api-access-55nd8\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.155642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.155503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.155642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.155554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.155642 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.155601 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4dfb11-bdce-43ec-a938-730bac5de22d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.156382 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.156356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a4dfb11-bdce-43ec-a938-730bac5de22d-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.158492 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.158423 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.158492 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.158442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a4dfb11-bdce-43ec-a938-730bac5de22d-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.169234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.169212 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55nd8\" (UniqueName: \"kubernetes.io/projected/3a4dfb11-bdce-43ec-a938-730bac5de22d-kube-api-access-55nd8\") pod \"prometheus-operator-567f7b8b7c-z529q\" (UID: \"3a4dfb11-bdce-43ec-a938-730bac5de22d\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:29.221205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:29.221170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" Jul 08 18:05:31.870648 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.870616 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-zvtlz"] Jul 08 18:05:31.884267 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.884249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:31.891380 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.891358 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zvtlz"] Jul 08 18:05:31.892652 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.892635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 18:05:31.892770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.892724 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 18:05:31.892916 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.892896 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2tzv5\"" Jul 08 18:05:31.979470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:31.979430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9gnt\" (UniqueName: \"kubernetes.io/projected/e4bcf3ba-96ea-4462-959b-dbbee92715d5-kube-api-access-s9gnt\") pod \"downloads-685bfb5f64-zvtlz\" (UID: \"e4bcf3ba-96ea-4462-959b-dbbee92715d5\") " pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:32.080307 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:32.080266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9gnt\" (UniqueName: \"kubernetes.io/projected/e4bcf3ba-96ea-4462-959b-dbbee92715d5-kube-api-access-s9gnt\") pod \"downloads-685bfb5f64-zvtlz\" (UID: \"e4bcf3ba-96ea-4462-959b-dbbee92715d5\") " pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:32.093663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:32.093631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9gnt\" (UniqueName: \"kubernetes.io/projected/e4bcf3ba-96ea-4462-959b-dbbee92715d5-kube-api-access-s9gnt\") pod \"downloads-685bfb5f64-zvtlz\" (UID: \"e4bcf3ba-96ea-4462-959b-dbbee92715d5\") " pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:32.192958 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:32.192930 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:32.567218 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:32.567190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-zvtlz"] Jul 08 18:05:32.571163 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:32.571130 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4bcf3ba_96ea_4462_959b_dbbee92715d5.slice/crio-dae3e50d5830f1b1082886dcae76730eea11f0091b16e8fb1a9583d2275173e2 WatchSource:0}: Error finding container dae3e50d5830f1b1082886dcae76730eea11f0091b16e8fb1a9583d2275173e2: Status 404 returned error can't find the container with id dae3e50d5830f1b1082886dcae76730eea11f0091b16e8fb1a9583d2275173e2 Jul 08 18:05:32.591654 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:32.591627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-z529q"] Jul 08 18:05:32.594860 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:32.594829 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4dfb11_bdce_43ec_a938_730bac5de22d.slice/crio-3aa83e48d53317df8c4cc39f29f657e8ea5c7d86acaf24058a9d2065310288bd WatchSource:0}: Error finding container 3aa83e48d53317df8c4cc39f29f657e8ea5c7d86acaf24058a9d2065310288bd: Status 404 returned error can't find the container with id 3aa83e48d53317df8c4cc39f29f657e8ea5c7d86acaf24058a9d2065310288bd Jul 08 18:05:33.403064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.402985 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgwh2" event={"ID":"32133831-f978-43c4-94fe-33bfe0490974","Type":"ContainerStarted","Data":"e55f2e360b816bd4bb049ee87bf94b20734d33da19e5e2cdf91f083f4f2f935d"} Jul 08 18:05:33.405331 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.405305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxlzr" event={"ID":"6039c0a5-514c-4508-8961-80a26b8ed4e6","Type":"ContainerStarted","Data":"4a4b4658fe7f6bba1b0417a288a36f2cb6d6d3d91eeafe48bbedcff99efbb12e"} Jul 08 18:05:33.405485 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.405356 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wxlzr" event={"ID":"6039c0a5-514c-4508-8961-80a26b8ed4e6","Type":"ContainerStarted","Data":"3ff55d67ecd266271dd33253595ca3219a35b7bd10d07bb709089efe1ee2273e"} Jul 08 18:05:33.405485 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.405444 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:33.407037 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.407005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" event={"ID":"3a4dfb11-bdce-43ec-a938-730bac5de22d","Type":"ContainerStarted","Data":"3aa83e48d53317df8c4cc39f29f657e8ea5c7d86acaf24058a9d2065310288bd"} Jul 08 18:05:33.410185 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.410166 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="4f938b27d6066366d28f67c2435b695f76848c00e5fe074ecf64b5062d878623" exitCode=0 Jul 08 18:05:33.410275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.410249 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"4f938b27d6066366d28f67c2435b695f76848c00e5fe074ecf64b5062d878623"} Jul 08 18:05:33.412240 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.412213 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zvtlz" event={"ID":"e4bcf3ba-96ea-4462-959b-dbbee92715d5","Type":"ContainerStarted","Data":"dae3e50d5830f1b1082886dcae76730eea11f0091b16e8fb1a9583d2275173e2"} Jul 08 18:05:33.414046 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.414023 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vlvs2" event={"ID":"2c54a90d-3ccc-4291-a248-2a72dcfc71b2","Type":"ContainerStarted","Data":"fb294bd70ea766d7d3bf4f8e02acd57de1a6f3c3043adf502fa68d37e00dcbdb"} Jul 08 18:05:33.475173 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.475129 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wxlzr" podStartSLOduration=2.652600065 podStartE2EDuration="10.475110668s" totalImagesPullingTime="7.822510603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:33.434372495 +0000 UTC m=+47.773816500" watchObservedRunningTime="2026-07-08 18:05:33.475110668 +0000 UTC m=+47.814554678" Jul 08 18:05:33.503236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:33.503107 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vlvs2" podStartSLOduration=1.641698533 podStartE2EDuration="9.503090983s" totalImagesPullingTime="7.86139245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:33.502529355 +0000 UTC m=+47.841973365" watchObservedRunningTime="2026-07-08 18:05:33.503090983 +0000 UTC m=+47.842534968" Jul 08 18:05:34.421243 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:34.420834 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd748a79-b017-4b08-8450-5d0734cefe9e" containerID="0d9ff0b781efaa132e583a8350884f3d92dc124bc8c04422eb6936fa90860138" exitCode=0 Jul 08 18:05:34.421969 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:34.421940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerDied","Data":"0d9ff0b781efaa132e583a8350884f3d92dc124bc8c04422eb6936fa90860138"} Jul 08 18:05:35.428018 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.427975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgwh2" event={"ID":"32133831-f978-43c4-94fe-33bfe0490974","Type":"ContainerStarted","Data":"411a87d70cdd003d6c3a4ba8b4c735a3bb6b784c4863b2131e6ed37174707196"} Jul 08 18:05:35.430098 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.430071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" event={"ID":"3a4dfb11-bdce-43ec-a938-730bac5de22d","Type":"ContainerStarted","Data":"61d177dd929124b0b5df4bdc4b772922142c3bbc97dd7480f1beedf39c67aff3"} Jul 08 18:05:35.430242 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.430122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" event={"ID":"3a4dfb11-bdce-43ec-a938-730bac5de22d","Type":"ContainerStarted","Data":"26d7d10b92f0d28f5a884f293c3aec631842b088749724a377b34964c2d035d6"} Jul 08 18:05:35.433867 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.433840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kplpd" event={"ID":"bd748a79-b017-4b08-8450-5d0734cefe9e","Type":"ContainerStarted","Data":"875967de618f757061dbc3bf7dac23fff253c05c0d4bc651d4a6696763405ff6"} Jul 08 18:05:35.498171 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.498117 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kplpd" podStartSLOduration=8.870545629 podStartE2EDuration="49.498103247s" totalImagesPullingTime="40.627557618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:35.495863658 +0000 UTC m=+49.835307666" watchObservedRunningTime="2026-07-08 18:05:35.498103247 +0000 UTC m=+49.837547255" Jul 08 18:05:35.498513 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:35.498477 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-z529q" podStartSLOduration=5.490747554 podStartE2EDuration="7.498448195s" totalImagesPullingTime="2.007700641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:35.45861568 +0000 UTC m=+49.798059687" watchObservedRunningTime="2026-07-08 18:05:35.498448195 +0000 UTC m=+49.837892205" Jul 08 18:05:37.536351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.536316 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q8285"] Jul 08 18:05:37.548978 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.548957 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.555877 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.555854 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xdrn7\"" Jul 08 18:05:37.555992 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.555893 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 18:05:37.555992 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.555901 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 18:05:37.556102 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.556019 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 18:05:37.725562 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jddng\" (UniqueName: \"kubernetes.io/projected/b5990daa-0737-479d-8df3-638a380392dc-kube-api-access-jddng\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-sys\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-root\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-metrics-client-ca\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-wtmp\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725961 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-textfile\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725961 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-tls\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725961 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.725961 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.725937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.826810 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-sys\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.826810 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-root\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.826810 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-metrics-client-ca\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.826810 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-wtmp\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826857 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-root\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826859 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-sys\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-textfile\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-tls\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.826970 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-wtmp\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.827034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.827062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jddng\" (UniqueName: \"kubernetes.io/projected/b5990daa-0737-479d-8df3-638a380392dc-kube-api-access-jddng\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827724 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.827181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-textfile\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827724 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.827409 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-metrics-client-ca\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.827724 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.827417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.829512 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.829487 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.829762 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.829744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5990daa-0737-479d-8df3-638a380392dc-node-exporter-tls\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.849673 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.849647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jddng\" (UniqueName: \"kubernetes.io/projected/b5990daa-0737-479d-8df3-638a380392dc-kube-api-access-jddng\") pod \"node-exporter-q8285\" (UID: \"b5990daa-0737-479d-8df3-638a380392dc\") " pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:37.857583 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:37.857553 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8285" Jul 08 18:05:38.448766 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:38.448724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xgwh2" event={"ID":"32133831-f978-43c4-94fe-33bfe0490974","Type":"ContainerStarted","Data":"7784be082a64c33bddcbaa0633bd8f62048a0d5f4c9404540c448996173fd916"} Jul 08 18:05:38.450036 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:38.450006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8285" event={"ID":"b5990daa-0737-479d-8df3-638a380392dc","Type":"ContainerStarted","Data":"73be281f8efae87dd4487a7d5f8842ff77b0500f9c528ca5e8e0a02e1d5f7b1a"} Jul 08 18:05:38.490376 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:38.490328 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xgwh2" podStartSLOduration=2.353609771 podStartE2EDuration="15.490314677s" totalImagesPullingTime="13.136704906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:05:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:38.488475886 +0000 UTC m=+52.827919916" watchObservedRunningTime="2026-07-08 18:05:38.490314677 +0000 UTC m=+52.829758685" Jul 08 18:05:38.974939 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:38.974904 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:05:39.005078 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.005052 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.012228 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.012192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:05:39.013627 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.013601 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 18:05:39.013984 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.013950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 18:05:39.014110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.014013 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 18:05:39.014593 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.014573 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 18:05:39.015087 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.014783 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 18:05:39.015324 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.015307 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 18:05:39.023658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.019056 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 18:05:39.023658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.019767 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 18:05:39.023658 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.020119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6vb8\"" Jul 08 18:05:39.025631 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.025609 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 18:05:39.034795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.034898 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.034898 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034839 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj6gz\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.034898 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.034987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035213 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035213 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035213 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.035213 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.035176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.135806 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.135806 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj6gz\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.135950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.136630 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.136605 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.138297 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.138268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.140135 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.140104 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.140339 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.140299 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.142624 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.142362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.142624 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.142584 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.142821 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.142655 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.142821 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.142661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.143224 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.143204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.143732 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.143688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.143958 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.143909 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.144645 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.144623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.156341 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.156319 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj6gz\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz\") pod \"alertmanager-main-0\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.317966 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.317896 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:05:39.485166 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:39.485069 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:05:39.560006 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:39.559976 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea3294d_ae1d_4f52_8888_d3bd251fa5f4.slice/crio-d226d632288c068cdc8609de092c30df918f7940512d86113254923918255300 WatchSource:0}: Error finding container d226d632288c068cdc8609de092c30df918f7940512d86113254923918255300: Status 404 returned error can't find the container with id d226d632288c068cdc8609de092c30df918f7940512d86113254923918255300 Jul 08 18:05:40.458416 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.458380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"d226d632288c068cdc8609de092c30df918f7940512d86113254923918255300"} Jul 08 18:05:40.460223 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.460185 2562 generic.go:356] "Generic (PLEG): container finished" podID="b5990daa-0737-479d-8df3-638a380392dc" containerID="e22585c4b9083d666e3458e696b28755cde4ac581ede6b2e791652b508ffcaef" exitCode=0 Jul 08 18:05:40.460325 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.460241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8285" event={"ID":"b5990daa-0737-479d-8df3-638a380392dc","Type":"ContainerDied","Data":"e22585c4b9083d666e3458e696b28755cde4ac581ede6b2e791652b508ffcaef"} Jul 08 18:05:40.735812 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.735725 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-78fdbb6986-6psdj"] Jul 08 18:05:40.738291 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.738267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746336 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746437 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746437 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746614 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746544 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f65tc\" (UniqueName: \"kubernetes.io/projected/f0ad3018-9d1e-4646-8884-93ef62050456-kube-api-access-f65tc\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746614 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-grpc-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.746697 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.746649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0ad3018-9d1e-4646-8884-93ef62050456-metrics-client-ca\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.747030 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.747015 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t7547\"" Jul 08 18:05:40.748545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.748349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 18:05:40.748545 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.748421 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-avoi9cnhc2okg\"" Jul 08 18:05:40.749121 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.749098 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 18:05:40.749228 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.749108 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 18:05:40.749637 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.749621 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 18:05:40.752040 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.752025 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 18:05:40.757627 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.757605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78fdbb6986-6psdj"] Jul 08 18:05:40.847041 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847041 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847227 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847227 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847227 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65tc\" (UniqueName: \"kubernetes.io/projected/f0ad3018-9d1e-4646-8884-93ef62050456-kube-api-access-f65tc\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847227 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-grpc-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847227 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0ad3018-9d1e-4646-8884-93ef62050456-metrics-client-ca\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.847450 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.847332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.848553 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.848512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f0ad3018-9d1e-4646-8884-93ef62050456-metrics-client-ca\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.849909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.849882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.850030 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.850007 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.850110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.850094 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.850267 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.850243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.850333 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.850307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-thanos-querier-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.850412 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.850392 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f0ad3018-9d1e-4646-8884-93ef62050456-secret-grpc-tls\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:40.857222 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:40.857200 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65tc\" (UniqueName: \"kubernetes.io/projected/f0ad3018-9d1e-4646-8884-93ef62050456-kube-api-access-f65tc\") pod \"thanos-querier-78fdbb6986-6psdj\" (UID: \"f0ad3018-9d1e-4646-8884-93ef62050456\") " pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:41.049653 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.049573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:41.399496 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.399443 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-78fdbb6986-6psdj"] Jul 08 18:05:41.438280 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:41.438256 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ad3018_9d1e_4646_8884_93ef62050456.slice/crio-25e88fef353c5d163b604da75b6b802ec3ad72d702e4a02a1d846877860a8058 WatchSource:0}: Error finding container 25e88fef353c5d163b604da75b6b802ec3ad72d702e4a02a1d846877860a8058: Status 404 returned error can't find the container with id 25e88fef353c5d163b604da75b6b802ec3ad72d702e4a02a1d846877860a8058 Jul 08 18:05:41.466447 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.466418 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118" exitCode=0 Jul 08 18:05:41.466783 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.466549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118"} Jul 08 18:05:41.468470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.468434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"25e88fef353c5d163b604da75b6b802ec3ad72d702e4a02a1d846877860a8058"} Jul 08 18:05:41.471006 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.470984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8285" event={"ID":"b5990daa-0737-479d-8df3-638a380392dc","Type":"ContainerStarted","Data":"0ef5585c37c17f381a254f4caca339cc45c7bf70d2f6d926d87a85ba5108da49"} Jul 08 18:05:41.471100 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.471025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8285" event={"ID":"b5990daa-0737-479d-8df3-638a380392dc","Type":"ContainerStarted","Data":"fd80d374421636fc2ff6b06ac6b8e704c85a777165901a57b5ba9e7d001aa2d3"} Jul 08 18:05:41.548444 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:41.548395 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q8285" podStartSLOduration=2.765103833 podStartE2EDuration="4.548381609s" totalImagesPullingTime="1.662626676s" totalInitContainerRuntime="120.6511ms" isStatefulPod=true podCreationTimestamp="2026-07-08 18:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:41.547672462 +0000 UTC m=+55.887116470" watchObservedRunningTime="2026-07-08 18:05:41.548381609 +0000 UTC m=+55.887825616" Jul 08 18:05:42.103557 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.103521 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv"] Jul 08 18:05:42.105754 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.105740 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.108553 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.108529 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-79h2j\"" Jul 08 18:05:42.108670 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.108594 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 18:05:42.118806 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.118691 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv"] Jul 08 18:05:42.161030 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.160993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.161203 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.161176 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9xkm\" (UniqueName: \"kubernetes.io/projected/40694806-ab82-4e2e-b797-b8a8a8c9730b-kube-api-access-d9xkm\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.265910 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.263427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9xkm\" (UniqueName: \"kubernetes.io/projected/40694806-ab82-4e2e-b797-b8a8a8c9730b-kube-api-access-d9xkm\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.265910 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.263520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.265910 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:42.263670 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 18:05:42.265910 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:05:42.263811 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert podName:40694806-ab82-4e2e-b797-b8a8a8c9730b nodeName:}" failed. No retries permitted until 2026-07-08 18:05:42.763791579 +0000 UTC m=+57.103235575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-tmbxv" (UID: "40694806-ab82-4e2e-b797-b8a8a8c9730b") : secret "monitoring-plugin-cert" not found Jul 08 18:05:42.282181 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.282154 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9xkm\" (UniqueName: \"kubernetes.io/projected/40694806-ab82-4e2e-b797-b8a8a8c9730b-kube-api-access-d9xkm\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.767197 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.767155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.770151 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.770125 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/40694806-ab82-4e2e-b797-b8a8a8c9730b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-tmbxv\" (UID: \"40694806-ab82-4e2e-b797-b8a8a8c9730b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:42.904078 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.904047 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w"] Jul 08 18:05:42.906246 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.906231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.910002 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.909980 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 18:05:42.910002 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.909996 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7xfp2\"" Jul 08 18:05:42.910247 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.910232 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 18:05:42.910302 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.910265 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 18:05:42.912324 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.912304 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 18:05:42.918517 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.918502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 18:05:42.925280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.925265 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 18:05:42.969581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-serving-certs-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969717 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969717 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-federate-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969717 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969888 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969888 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969888 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-metrics-client-ca\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.969888 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.969842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxzz\" (UniqueName: \"kubernetes.io/projected/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-kube-api-access-kbxzz\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:42.976813 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:42.976778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w"] Jul 08 18:05:43.016809 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.016546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:43.071073 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.070984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071073 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-federate-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071073 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-metrics-client-ca\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxzz\" (UniqueName: \"kubernetes.io/projected/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-kube-api-access-kbxzz\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.071351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.071298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-serving-certs-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.072158 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.072137 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-serving-certs-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.072497 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.072473 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-metrics-client-ca\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.072929 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.072908 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.076021 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.075976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.076121 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.076055 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-federate-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.076929 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.076876 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-telemeter-client-tls\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.077280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.077234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.084647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.084606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxzz\" (UniqueName: \"kubernetes.io/projected/9ef0a0b4-e098-44d2-b33e-dca6bc120f0a-kube-api-access-kbxzz\") pod \"telemeter-client-7d7b9f98b7-lnq6w\" (UID: \"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a\") " pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.175482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.175436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv"] Jul 08 18:05:43.179855 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:43.179835 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40694806_ab82_4e2e_b797_b8a8a8c9730b.slice/crio-bc9ea6d7d22af1f889eb26f88bcb7557344b6ff217375dc4771ca1afbe1d9b0b WatchSource:0}: Error finding container bc9ea6d7d22af1f889eb26f88bcb7557344b6ff217375dc4771ca1afbe1d9b0b: Status 404 returned error can't find the container with id bc9ea6d7d22af1f889eb26f88bcb7557344b6ff217375dc4771ca1afbe1d9b0b Jul 08 18:05:43.214967 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.214940 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" Jul 08 18:05:43.382931 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.382892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w"] Jul 08 18:05:43.387358 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:43.387330 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef0a0b4_e098_44d2_b33e_dca6bc120f0a.slice/crio-ffa84762cf7c1e9d310a412c30cbba15c78e8ac9f99d5b6afa663246d2702910 WatchSource:0}: Error finding container ffa84762cf7c1e9d310a412c30cbba15c78e8ac9f99d5b6afa663246d2702910: Status 404 returned error can't find the container with id ffa84762cf7c1e9d310a412c30cbba15c78e8ac9f99d5b6afa663246d2702910 Jul 08 18:05:43.424481 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.423884 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wxlzr" Jul 08 18:05:43.482616 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.482581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" event={"ID":"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a","Type":"ContainerStarted","Data":"ffa84762cf7c1e9d310a412c30cbba15c78e8ac9f99d5b6afa663246d2702910"} Jul 08 18:05:43.484184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:43.484159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" event={"ID":"40694806-ab82-4e2e-b797-b8a8a8c9730b","Type":"ContainerStarted","Data":"bc9ea6d7d22af1f889eb26f88bcb7557344b6ff217375dc4771ca1afbe1d9b0b"} Jul 08 18:05:44.115543 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.115518 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:05:44.120196 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.120104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.129666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.125742 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bg1h9av3lduh6\"" Jul 08 18:05:44.129666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.126121 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 18:05:44.129666 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.128701 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 18:05:44.130948 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.130931 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 18:05:44.131896 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.130960 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 18:05:44.132189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131084 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 18:05:44.132358 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131128 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wx8zb\"" Jul 08 18:05:44.132641 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131127 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131199 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131197 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131263 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131311 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131349 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.131355 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 18:05:44.137139 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.134556 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 18:05:44.137801 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.137436 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 18:05:44.138138 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.138034 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 18:05:44.149421 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.149398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:05:44.183000 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.182974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183136 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183181 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbhrb\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183687 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.183761 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.184060 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.183793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.284924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.284856 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbhrb\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285528 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285509 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285617 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285617 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.285814 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285776 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286147 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286147 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286147 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285909 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286147 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.285952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286890 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.286544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.286890 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.286669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.288378 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.288355 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.290081 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.289221 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.290081 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.289286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.290081 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.289752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.290081 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.290020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.292319 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.292246 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.293237 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.293186 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.293862 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.293824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.293954 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.293937 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.296640 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.296592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbhrb\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.297003 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.296942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.297137 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.297101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.297676 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.297610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.297676 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.297634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.298728 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.298199 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.298728 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.298692 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.299602 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.299565 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.443432 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.443369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:05:44.500886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.500562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b"} Jul 08 18:05:44.500886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.500648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53"} Jul 08 18:05:44.500886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.500693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08"} Jul 08 18:05:44.500886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.500732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4"} Jul 08 18:05:44.500886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.500773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195"} Jul 08 18:05:44.639762 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:44.639672 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:05:45.345954 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:45.345923 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jkq98" Jul 08 18:05:45.780216 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:45.780181 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9c7e23_46ee_4fd6_92d3_d25cddf7ada0.slice/crio-f845526abb6b775f212dafcaddfe96c12bb6ce69dfab32b853c20c45d3b81ca6 WatchSource:0}: Error finding container f845526abb6b775f212dafcaddfe96c12bb6ce69dfab32b853c20c45d3b81ca6: Status 404 returned error can't find the container with id f845526abb6b775f212dafcaddfe96c12bb6ce69dfab32b853c20c45d3b81ca6 Jul 08 18:05:46.515299 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:46.515264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"f845526abb6b775f212dafcaddfe96c12bb6ce69dfab32b853c20c45d3b81ca6"} Jul 08 18:05:47.521131 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.521104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" event={"ID":"40694806-ab82-4e2e-b797-b8a8a8c9730b","Type":"ContainerStarted","Data":"14150d3a41543be4d8a1a6cd25f8b80820d9b9bc42c058da07bb9ca22f5f90af"} Jul 08 18:05:47.521587 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.521298 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:47.523931 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.523903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" event={"ID":"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a","Type":"ContainerStarted","Data":"f44d727da158943f7d03e067bfc160ac1af2172f3a42cac49ef7fa0d53806261"} Jul 08 18:05:47.524034 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.523965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" event={"ID":"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a","Type":"ContainerStarted","Data":"5971148c9a64558332ebf7ef955161d037fb0148bece15a3f16e15baeb2ad899"} Jul 08 18:05:47.524034 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.523997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" event={"ID":"9ef0a0b4-e098-44d2-b33e-dca6bc120f0a","Type":"ContainerStarted","Data":"5548bcc32d3b864f756b341f982ee3caefd68a4672f1ecd56b77acce7d163c93"} Jul 08 18:05:47.526971 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.526948 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" Jul 08 18:05:47.528295 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.528235 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerStarted","Data":"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322"} Jul 08 18:05:47.530561 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.530542 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" exitCode=0 Jul 08 18:05:47.530641 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.530618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} Jul 08 18:05:47.533705 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.533685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"c9e6a6d4fc8dd93bdd514ab93d005b616e6baff4e3786186e820e752491a1e2f"} Jul 08 18:05:47.533795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.533746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"8d4a02c5a63a38f1a7efb797f2f869b54640632dbbe005fcf33ca34e3efd91ae"} Jul 08 18:05:47.533795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.533779 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"0e888a959f466b75bc8876f2e8640b8026048ba945b2bf93f6fb4148953a6f1c"} Jul 08 18:05:47.533863 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.533816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"7e957ba7bc3097862e9f8eae8395ad31635498fc8b28e41c91434361fc351b99"} Jul 08 18:05:47.533863 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.533857 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"6e75b4cc15e6adf89ad86fd0f439b2cc8beb4a516c2d061216c8525955c18dc4"} Jul 08 18:05:47.581718 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.581679 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-tmbxv" podStartSLOduration=1.67703067 podStartE2EDuration="5.581665691s" totalImagesPullingTime="3.904635021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:47.551667994 +0000 UTC m=+61.891112003" watchObservedRunningTime="2026-07-08 18:05:47.581665691 +0000 UTC m=+61.921109699" Jul 08 18:05:47.663349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.663287 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.630397818 podStartE2EDuration="9.663264814s" totalImagesPullingTime="7.017652979s" totalInitContainerRuntime="15.214017ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:47.659035695 +0000 UTC m=+61.998479705" watchObservedRunningTime="2026-07-08 18:05:47.663264814 +0000 UTC m=+62.002708825" Jul 08 18:05:47.734100 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:47.734044 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d7b9f98b7-lnq6w" podStartSLOduration=2.037116793 podStartE2EDuration="5.734026985s" totalImagesPullingTime="3.696910192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:47.733038307 +0000 UTC m=+62.072482341" watchObservedRunningTime="2026-07-08 18:05:47.734026985 +0000 UTC m=+62.073470993" Jul 08 18:05:48.543520 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:48.543478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" event={"ID":"f0ad3018-9d1e-4646-8884-93ef62050456","Type":"ContainerStarted","Data":"c84eba505ca03f95a2f72a180f053e92dcd87022223a2968d6d8bebf58bcafe7"} Jul 08 18:05:48.544102 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:48.544081 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:48.581566 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:48.581510 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" podStartSLOduration=2.935114306 podStartE2EDuration="8.581493593s" totalImagesPullingTime="5.646379287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:48.579994682 +0000 UTC m=+62.919438717" watchObservedRunningTime="2026-07-08 18:05:48.581493593 +0000 UTC m=+62.920937602" Jul 08 18:05:51.967145 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.967097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:51.967575 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.967166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:51.972056 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.972033 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:05:51.972771 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.972751 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:05:51.984477 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.981002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b80a286-c8d8-4fe3-a6c7-103281e1dce7-metrics-certs\") pod \"network-metrics-daemon-b6q6h\" (UID: \"2b80a286-c8d8-4fe3-a6c7-103281e1dce7\") " pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:51.984477 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:51.981044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d6085b44-3dca-430b-a48d-0b8c9dcc8942-original-pull-secret\") pod \"global-pull-secret-syncer-2tv7h\" (UID: \"d6085b44-3dca-430b-a48d-0b8c9dcc8942\") " pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:52.068158 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.068125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:52.071284 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.071257 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:05:52.081842 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.081808 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:05:52.091990 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.091967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2tv7h" Jul 08 18:05:52.092389 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.092374 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbv88\" (UniqueName: \"kubernetes.io/projected/a1995353-4b17-4deb-a76c-6bba7b344da0-kube-api-access-tbv88\") pod \"network-check-target-8mzhb\" (UID: \"a1995353-4b17-4deb-a76c-6bba7b344da0\") " pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:52.101745 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.101724 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8pfq\"" Jul 08 18:05:52.105587 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.105572 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9s8r\"" Jul 08 18:05:52.108019 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.108000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6q6h" Jul 08 18:05:52.113338 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:52.113314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:05:54.554835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:54.554805 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-78fdbb6986-6psdj" Jul 08 18:05:57.776435 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:57.776364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8mzhb"] Jul 08 18:05:57.779999 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:57.779968 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1995353_4b17_4deb_a76c_6bba7b344da0.slice/crio-dcf8ed7372aee769dd40913b274d201df2840e02bda82170e974bea7549cd4de WatchSource:0}: Error finding container dcf8ed7372aee769dd40913b274d201df2840e02bda82170e974bea7549cd4de: Status 404 returned error can't find the container with id dcf8ed7372aee769dd40913b274d201df2840e02bda82170e974bea7549cd4de Jul 08 18:05:58.002939 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.002911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2tv7h"] Jul 08 18:05:58.005991 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:58.005960 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6085b44_3dca_430b_a48d_0b8c9dcc8942.slice/crio-cdda798af3298ef6d2099062c605502c61e39e02be3f346ea9850b78a714fa6a WatchSource:0}: Error finding container cdda798af3298ef6d2099062c605502c61e39e02be3f346ea9850b78a714fa6a: Status 404 returned error can't find the container with id cdda798af3298ef6d2099062c605502c61e39e02be3f346ea9850b78a714fa6a Jul 08 18:05:58.008045 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.008017 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6q6h"] Jul 08 18:05:58.010486 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:05:58.010382 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b80a286_c8d8_4fe3_a6c7_103281e1dce7.slice/crio-74cdbdffd0d5c8cc3bdd7988158f467844065b23c03081da1ba11896742ebc7e WatchSource:0}: Error finding container 74cdbdffd0d5c8cc3bdd7988158f467844065b23c03081da1ba11896742ebc7e: Status 404 returned error can't find the container with id 74cdbdffd0d5c8cc3bdd7988158f467844065b23c03081da1ba11896742ebc7e Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595948 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} Jul 08 18:05:58.595975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.595983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerStarted","Data":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} Jul 08 18:05:58.597473 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.597423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2tv7h" event={"ID":"d6085b44-3dca-430b-a48d-0b8c9dcc8942","Type":"ContainerStarted","Data":"cdda798af3298ef6d2099062c605502c61e39e02be3f346ea9850b78a714fa6a"} Jul 08 18:05:58.599338 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.599273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-zvtlz" event={"ID":"e4bcf3ba-96ea-4462-959b-dbbee92715d5","Type":"ContainerStarted","Data":"e3f9ba67e07090049a78971d116fc0869b83039b2679a882baef298e9880cd47"} Jul 08 18:05:58.599746 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.599726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:58.600793 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.600769 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6q6h" event={"ID":"2b80a286-c8d8-4fe3-a6c7-103281e1dce7","Type":"ContainerStarted","Data":"74cdbdffd0d5c8cc3bdd7988158f467844065b23c03081da1ba11896742ebc7e"} Jul 08 18:05:58.602158 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.602137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8mzhb" event={"ID":"a1995353-4b17-4deb-a76c-6bba7b344da0","Type":"ContainerStarted","Data":"dcf8ed7372aee769dd40913b274d201df2840e02bda82170e974bea7549cd4de"} Jul 08 18:05:58.607126 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.607103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-zvtlz" Jul 08 18:05:58.680590 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.680537 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-zvtlz" podStartSLOduration=2.521728852 podStartE2EDuration="27.680516674s" totalImagesPullingTime="25.158787822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:58.680037636 +0000 UTC m=+73.019481643" watchObservedRunningTime="2026-07-08 18:05:58.680516674 +0000 UTC m=+73.019960685" Jul 08 18:05:58.692053 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:58.691993 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.580561676 podStartE2EDuration="14.691972861s" totalImagesPullingTime="10.081163658s" totalInitContainerRuntime="30.247527ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:58.648623611 +0000 UTC m=+72.988067632" watchObservedRunningTime="2026-07-08 18:05:58.691972861 +0000 UTC m=+73.031416872" Jul 08 18:05:59.444255 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:05:59.444209 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:06:01.623777 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:01.623721 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6q6h" event={"ID":"2b80a286-c8d8-4fe3-a6c7-103281e1dce7","Type":"ContainerStarted","Data":"0e5c5f3b28af6b14af92cf63733ec78b9e89deb8c4e10769c3e9b444e57df017"} Jul 08 18:06:01.624291 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:01.623836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6q6h" event={"ID":"2b80a286-c8d8-4fe3-a6c7-103281e1dce7","Type":"ContainerStarted","Data":"2d9f9d12520c18722bc3b75f11868eb85df0e51bec614949aa258b7c69599157"} Jul 08 18:06:01.647688 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:01.646076 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b6q6h" podStartSLOduration=72.754320567 podStartE2EDuration="1m15.646053967s" totalImagesPullingTime="2.8917334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:06:01.644381731 +0000 UTC m=+75.983825740" watchObservedRunningTime="2026-07-08 18:06:01.646053967 +0000 UTC m=+75.985497978" Jul 08 18:06:05.647505 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:05.647441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2tv7h" event={"ID":"d6085b44-3dca-430b-a48d-0b8c9dcc8942","Type":"ContainerStarted","Data":"e6b175ef5147bb1a3069e93b9f0bc0a1b58b91e325eae30a6bc56902205378e5"} Jul 08 18:06:05.649279 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:05.649238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8mzhb" event={"ID":"a1995353-4b17-4deb-a76c-6bba7b344da0","Type":"ContainerStarted","Data":"ce37ec5d1f92cd7b16382554bf436987989b68139b7323ad4433d58979f3ab98"} Jul 08 18:06:05.649413 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:05.649385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:06:05.705723 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:05.705665 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2tv7h" podStartSLOduration=72.673760653 podStartE2EDuration="1m19.705645925s" totalImagesPullingTime="7.031885272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:06:05.670990443 +0000 UTC m=+80.010434450" watchObservedRunningTime="2026-07-08 18:06:05.705645925 +0000 UTC m=+80.045089933" Jul 08 18:06:05.705910 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:05.705886 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8mzhb" podStartSLOduration=72.459604599 podStartE2EDuration="1m19.705878026s" totalImagesPullingTime="7.246273427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:06:05.704583506 +0000 UTC m=+80.044027515" watchObservedRunningTime="2026-07-08 18:06:05.705878026 +0000 UTC m=+80.045322034" Jul 08 18:06:10.279630 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:10.279488 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vlvs2_2c54a90d-3ccc-4291-a248-2a72dcfc71b2/serve-healthcheck-canary/0.log" Jul 08 18:06:36.656913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:36.656881 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8mzhb" Jul 08 18:06:54.616482 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:54.616432 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:06:54.635400 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:54.635375 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:06:54.855084 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:06:54.855053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:08.491739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.491698 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.491986 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="alertmanager" containerID="cri-o://a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" gracePeriod=120 Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.492077 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-metric" containerID="cri-o://344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" gracePeriod=120 Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.492101 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="prom-label-proxy" containerID="cri-o://55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" gracePeriod=120 Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.492142 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy" containerID="cri-o://7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" gracePeriod=120 Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.492171 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="config-reloader" containerID="cri-o://a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" gracePeriod=120 Jul 08 18:07:08.492234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.492130 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-web" containerID="cri-o://0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" gracePeriod=120 Jul 08 18:07:08.893644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893566 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" exitCode=0 Jul 08 18:07:08.893644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893590 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" exitCode=0 Jul 08 18:07:08.893644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893598 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" exitCode=0 Jul 08 18:07:08.893644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893605 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" exitCode=0 Jul 08 18:07:08.893873 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893647 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322"} Jul 08 18:07:08.893873 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53"} Jul 08 18:07:08.893873 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4"} Jul 08 18:07:08.893873 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:08.893825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195"} Jul 08 18:07:09.732140 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.732118 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:09.740577 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740559 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740587 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740606 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740627 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740644 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj6gz\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740664 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740695 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740725 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740761 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740791 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740822 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.740864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740855 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.741372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.740893 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle\") pod \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\" (UID: \"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4\") " Jul 08 18:07:09.742523 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.741741 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:09.742523 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.742322 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:09.742523 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.742480 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:09.744660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.743946 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume" (OuterVolumeSpecName: "config-volume") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.744660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.744269 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz" (OuterVolumeSpecName: "kube-api-access-kj6gz") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "kube-api-access-kj6gz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:07:09.744660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.744392 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.744660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.744422 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:07:09.744884 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.744756 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.745362 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.745339 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.745513 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.745486 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.745581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.745546 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out" (OuterVolumeSpecName: "config-out") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:09.748661 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.748637 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.755062 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.755039 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config" (OuterVolumeSpecName: "web-config") pod "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" (UID: "4ea3294d-ae1d-4f52-8888-d3bd251fa5f4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:09.842216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842137 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-main-tls\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842175 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-tls-assets\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842188 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-volume\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842200 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-cluster-tls-config\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842212 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj6gz\" (UniqueName: \"kubernetes.io/projected/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-kube-api-access-kj6gz\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842226 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842241 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-config-out\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842252 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-main-db\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842265 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-web-config\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842277 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-metrics-client-ca\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842288 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842303 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.842470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.842318 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:09.900178 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900141 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" exitCode=0 Jul 08 18:07:09.900178 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900174 2562 generic.go:356] "Generic (PLEG): container finished" podID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerID="0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" exitCode=0 Jul 08 18:07:09.900349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900225 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b"} Jul 08 18:07:09.900349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08"} Jul 08 18:07:09.900349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900238 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:09.900349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900328 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4ea3294d-ae1d-4f52-8888-d3bd251fa5f4","Type":"ContainerDied","Data":"d226d632288c068cdc8609de092c30df918f7940512d86113254923918255300"} Jul 08 18:07:09.900493 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.900376 2562 scope.go:117] "RemoveContainer" containerID="55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" Jul 08 18:07:09.908770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.908751 2562 scope.go:117] "RemoveContainer" containerID="344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" Jul 08 18:07:09.915501 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.915484 2562 scope.go:117] "RemoveContainer" containerID="7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" Jul 08 18:07:09.921951 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.921936 2562 scope.go:117] "RemoveContainer" containerID="0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" Jul 08 18:07:09.928196 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.928177 2562 scope.go:117] "RemoveContainer" containerID="a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" Jul 08 18:07:09.932423 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.932404 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:09.935558 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.935540 2562 scope.go:117] "RemoveContainer" containerID="a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" Jul 08 18:07:09.939069 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.939050 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:09.942996 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.942976 2562 scope.go:117] "RemoveContainer" containerID="7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118" Jul 08 18:07:09.949556 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.949540 2562 scope.go:117] "RemoveContainer" containerID="55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" Jul 08 18:07:09.949802 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.949781 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322\": container with ID starting with 55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322 not found: ID does not exist" containerID="55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" Jul 08 18:07:09.949851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.949812 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322"} err="failed to get container status \"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322\": rpc error: code = NotFound desc = could not find container \"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322\": container with ID starting with 55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322 not found: ID does not exist" Jul 08 18:07:09.949851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.949842 2562 scope.go:117] "RemoveContainer" containerID="344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" Jul 08 18:07:09.950076 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.950060 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b\": container with ID starting with 344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b not found: ID does not exist" containerID="344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" Jul 08 18:07:09.950111 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950082 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b"} err="failed to get container status \"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b\": rpc error: code = NotFound desc = could not find container \"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b\": container with ID starting with 344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b not found: ID does not exist" Jul 08 18:07:09.950111 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950097 2562 scope.go:117] "RemoveContainer" containerID="7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" Jul 08 18:07:09.950321 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.950305 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53\": container with ID starting with 7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53 not found: ID does not exist" containerID="7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" Jul 08 18:07:09.950356 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950326 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53"} err="failed to get container status \"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53\": rpc error: code = NotFound desc = could not find container \"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53\": container with ID starting with 7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53 not found: ID does not exist" Jul 08 18:07:09.950356 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950339 2562 scope.go:117] "RemoveContainer" containerID="0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" Jul 08 18:07:09.950559 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.950543 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08\": container with ID starting with 0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08 not found: ID does not exist" containerID="0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" Jul 08 18:07:09.950610 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950564 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08"} err="failed to get container status \"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08\": rpc error: code = NotFound desc = could not find container \"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08\": container with ID starting with 0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08 not found: ID does not exist" Jul 08 18:07:09.950610 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950576 2562 scope.go:117] "RemoveContainer" containerID="a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" Jul 08 18:07:09.950807 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.950791 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4\": container with ID starting with a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4 not found: ID does not exist" containerID="a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" Jul 08 18:07:09.950851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950820 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4"} err="failed to get container status \"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4\": rpc error: code = NotFound desc = could not find container \"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4\": container with ID starting with a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4 not found: ID does not exist" Jul 08 18:07:09.950851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.950833 2562 scope.go:117] "RemoveContainer" containerID="a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" Jul 08 18:07:09.951048 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.951034 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195\": container with ID starting with a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195 not found: ID does not exist" containerID="a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" Jul 08 18:07:09.951090 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951051 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195"} err="failed to get container status \"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195\": rpc error: code = NotFound desc = could not find container \"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195\": container with ID starting with a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195 not found: ID does not exist" Jul 08 18:07:09.951090 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951062 2562 scope.go:117] "RemoveContainer" containerID="7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118" Jul 08 18:07:09.951283 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:09.951269 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118\": container with ID starting with 7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118 not found: ID does not exist" containerID="7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118" Jul 08 18:07:09.951321 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951286 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118"} err="failed to get container status \"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118\": rpc error: code = NotFound desc = could not find container \"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118\": container with ID starting with 7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118 not found: ID does not exist" Jul 08 18:07:09.951321 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951298 2562 scope.go:117] "RemoveContainer" containerID="55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322" Jul 08 18:07:09.951525 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951504 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322"} err="failed to get container status \"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322\": rpc error: code = NotFound desc = could not find container \"55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322\": container with ID starting with 55decf16848e6889265a95302049c2f929f16895d3183054ef9655e5dde46322 not found: ID does not exist" Jul 08 18:07:09.951604 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951528 2562 scope.go:117] "RemoveContainer" containerID="344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b" Jul 08 18:07:09.951759 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951743 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b"} err="failed to get container status \"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b\": rpc error: code = NotFound desc = could not find container \"344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b\": container with ID starting with 344a5bbc28ff1ab1dbd26e3e0989675c81f3b7636fda5ce300ee3ae2087f9b2b not found: ID does not exist" Jul 08 18:07:09.951815 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951760 2562 scope.go:117] "RemoveContainer" containerID="7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53" Jul 08 18:07:09.951981 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951963 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53"} err="failed to get container status \"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53\": rpc error: code = NotFound desc = could not find container \"7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53\": container with ID starting with 7f85b6964c36c6648ac205eaf9e9a3abbe2beee905fa2adf866bc3a196505a53 not found: ID does not exist" Jul 08 18:07:09.952042 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.951981 2562 scope.go:117] "RemoveContainer" containerID="0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08" Jul 08 18:07:09.952197 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952175 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08"} err="failed to get container status \"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08\": rpc error: code = NotFound desc = could not find container \"0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08\": container with ID starting with 0b546f85c360314860c09cdadfef00eaa666d9cb8f2a4bb96995e51a85f4ec08 not found: ID does not exist" Jul 08 18:07:09.952277 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952200 2562 scope.go:117] "RemoveContainer" containerID="a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4" Jul 08 18:07:09.952410 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952394 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4"} err="failed to get container status \"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4\": rpc error: code = NotFound desc = could not find container \"a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4\": container with ID starting with a0aa1caee4e3ebc591a3d035ca4e2d5917807f72be8fef92f98fc244297413c4 not found: ID does not exist" Jul 08 18:07:09.952479 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952409 2562 scope.go:117] "RemoveContainer" containerID="a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195" Jul 08 18:07:09.952653 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952636 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195"} err="failed to get container status \"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195\": rpc error: code = NotFound desc = could not find container \"a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195\": container with ID starting with a20b48e814dc86cfdc592918ab29282724c2fd77e660e401e22a1fb5b66e4195 not found: ID does not exist" Jul 08 18:07:09.952720 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952653 2562 scope.go:117] "RemoveContainer" containerID="7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118" Jul 08 18:07:09.952857 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.952838 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118"} err="failed to get container status \"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118\": rpc error: code = NotFound desc = could not find container \"7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118\": container with ID starting with 7fde30076a68aeb21a394915db43df2d1ca38322634e000363a4ae34f9a1b118 not found: ID does not exist" Jul 08 18:07:09.973894 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.973872 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:09.974142 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974131 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-metric" Jul 08 18:07:09.974184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974145 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-metric" Jul 08 18:07:09.974184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974155 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="init-config-reloader" Jul 08 18:07:09.974184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974161 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="init-config-reloader" Jul 08 18:07:09.974184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974171 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="config-reloader" Jul 08 18:07:09.974184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974177 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="config-reloader" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974189 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="prom-label-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974195 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="prom-label-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974201 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="alertmanager" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974206 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="alertmanager" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974212 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974217 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974223 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-web" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974228 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-web" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974271 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="prom-label-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974290 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="alertmanager" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974298 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="config-reloader" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974303 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974310 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-web" Jul 08 18:07:09.974322 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.974317 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" containerName="kube-rbac-proxy-metric" Jul 08 18:07:09.979449 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.977835 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:09.981291 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.981274 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6vb8\"" Jul 08 18:07:09.981433 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.981420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 18:07:09.981689 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.981676 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 18:07:09.981986 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.981970 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 18:07:09.982034 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.981987 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 18:07:09.982281 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.982263 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 18:07:09.982357 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.982281 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 18:07:09.982538 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.982513 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 18:07:09.982538 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.982531 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 18:07:09.988122 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:09.988101 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 18:07:10.003157 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.003132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:10.043154 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043154 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043312 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-config-volume\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043372 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043338 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-config-out\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrcs2\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-kube-api-access-mrcs2\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-web-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.043612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.043547 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144123 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144123 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144123 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144348 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144348 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144447 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144542 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-config-volume\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144695 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144750 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-config-out\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.144806 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrcs2\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-kube-api-access-mrcs2\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-web-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144874 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.144911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145804 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.145773 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.145913 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.145835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63d67666-7d25-4f64-ac32-da1b600c573c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.147606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.147560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.147715 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.147604 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.147715 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.147660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.147822 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.147714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-config-volume\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.148062 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.148044 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/63d67666-7d25-4f64-ac32-da1b600c573c-config-out\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.148167 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.148152 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-web-config\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.148398 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.148378 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.148553 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.148538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.149513 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.149496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/63d67666-7d25-4f64-ac32-da1b600c573c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.156475 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.156434 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrcs2\" (UniqueName: \"kubernetes.io/projected/63d67666-7d25-4f64-ac32-da1b600c573c-kube-api-access-mrcs2\") pod \"alertmanager-main-0\" (UID: \"63d67666-7d25-4f64-ac32-da1b600c573c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.188333 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.188300 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ea3294d-ae1d-4f52-8888-d3bd251fa5f4" path="/var/lib/kubelet/pods/4ea3294d-ae1d-4f52-8888-d3bd251fa5f4/volumes" Jul 08 18:07:10.289270 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.289233 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 18:07:10.434586 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.434564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 18:07:10.436388 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:07:10.436364 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d67666_7d25_4f64_ac32_da1b600c573c.slice/crio-50a82bc64a4bfd71024dc7e5f21afbc61570feb91db0c38adde700009827db90 WatchSource:0}: Error finding container 50a82bc64a4bfd71024dc7e5f21afbc61570feb91db0c38adde700009827db90: Status 404 returned error can't find the container with id 50a82bc64a4bfd71024dc7e5f21afbc61570feb91db0c38adde700009827db90 Jul 08 18:07:10.905438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.905403 2562 generic.go:356] "Generic (PLEG): container finished" podID="63d67666-7d25-4f64-ac32-da1b600c573c" containerID="8bb2c4f5baa722dfb20f6a31f6809390d5906bd5e4914dca3bbaa95364d1cbfc" exitCode=0 Jul 08 18:07:10.905851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.905492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerDied","Data":"8bb2c4f5baa722dfb20f6a31f6809390d5906bd5e4914dca3bbaa95364d1cbfc"} Jul 08 18:07:10.905851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:10.905566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"50a82bc64a4bfd71024dc7e5f21afbc61570feb91db0c38adde700009827db90"} Jul 08 18:07:11.914415 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"82b7dd6da25027e75d706a0d2a99829f684bc173b3c2594bba937b1efd48cd0d"} Jul 08 18:07:11.914807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914447 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"2acd2ecd3fc55a02991fd74ac2541c987655b6db7ab83b636c94cf0ef839ea85"} Jul 08 18:07:11.914807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914493 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"f12908ae04436a3565963f9fff369a702059b8f61d4a848a542014ceeb0bed5b"} Jul 08 18:07:11.914807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"81cddc86f182809b3cd2847b9817239f1e0e551ee15d2e7f3d3f9c828d764678"} Jul 08 18:07:11.914807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914543 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"6be16101a73da470550821a5c27bc6987edf0d192ea33c0760341506681febf1"} Jul 08 18:07:11.914807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.914567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"63d67666-7d25-4f64-ac32-da1b600c573c","Type":"ContainerStarted","Data":"7e43cf9c655c5ecb7af39b5aeb0cdbf812d97e0153a05411205c9afabc04884b"} Jul 08 18:07:11.950901 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:11.950854 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.935474772 podStartE2EDuration="2.950839373s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.364601ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:07:11.950250105 +0000 UTC m=+146.289694126" watchObservedRunningTime="2026-07-08 18:07:11.950839373 +0000 UTC m=+146.290283407" Jul 08 18:07:12.849742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.849708 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:12.850349 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.850307 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="prometheus" containerID="cri-o://1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" gracePeriod=600 Jul 08 18:07:12.850770 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.850739 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-thanos" containerID="cri-o://62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" gracePeriod=600 Jul 08 18:07:12.850895 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.850874 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy" containerID="cri-o://1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" gracePeriod=600 Jul 08 18:07:12.850978 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.850960 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-web" containerID="cri-o://798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" gracePeriod=600 Jul 08 18:07:12.851061 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.851043 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="thanos-sidecar" containerID="cri-o://1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" gracePeriod=600 Jul 08 18:07:12.851137 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:12.851120 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="config-reloader" containerID="cri-o://9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" gracePeriod=600 Jul 08 18:07:13.107681 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.107624 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:13.171064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171034 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171075 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171100 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171125 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171148 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171172 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171196 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171230 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171269 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171315 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171343 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171381 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171487 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171520 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171545 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171537 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.171606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171577 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171636 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171665 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbhrb\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171690 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs\") pod \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\" (UID: \"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0\") " Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171794 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171981 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-metrics-client-ca\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.171982 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.172064 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.172000 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.172578 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.172243 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.174092 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.173995 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.174756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.174640 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:07:13.174993 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.174931 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.175183 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.175154 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.175264 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.175197 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.175973 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.175935 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.176337 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.176314 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.176496 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.176441 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.176944 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.176766 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:13.176944 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.176834 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 18:07:13.177758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.177699 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb" (OuterVolumeSpecName: "kube-api-access-jbhrb") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "kube-api-access-jbhrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:07:13.177886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.177857 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.178292 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.178252 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.178924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.178899 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config" (OuterVolumeSpecName: "config") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.179249 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.179228 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out" (OuterVolumeSpecName: "config-out") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:07:13.189816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.189787 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config" (OuterVolumeSpecName: "web-config") pod "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" (UID: "1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:07:13.272998 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.272969 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-tls-assets\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.272998 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.272992 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-kube-rbac-proxy\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.272998 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273002 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-grpc-tls\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273013 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273023 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273030 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273038 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config-out\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273046 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-config\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273053 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273061 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbhrb\" (UniqueName: \"kubernetes.io/projected/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-kube-api-access-jbhrb\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273069 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-metrics-client-certs\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273077 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-web-config\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273085 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273093 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273101 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273109 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273118 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-prometheus-k8s-db\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.273205 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.273126 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:07:13.926612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926582 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" exitCode=0 Jul 08 18:07:13.926612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926606 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" exitCode=0 Jul 08 18:07:13.926612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926612 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" exitCode=0 Jul 08 18:07:13.926612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926618 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" exitCode=0 Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926624 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" exitCode=0 Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926631 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" exitCode=0 Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926656 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926660 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926810 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926830 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926873 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} Jul 08 18:07:13.926917 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} Jul 08 18:07:13.927287 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.926955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0","Type":"ContainerDied","Data":"f845526abb6b775f212dafcaddfe96c12bb6ce69dfab32b853c20c45d3b81ca6"} Jul 08 18:07:13.934969 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.934951 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.941628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.941612 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.948130 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.948110 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.951955 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.951901 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:13.954880 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.954861 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.959103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.959076 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:13.962192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.962178 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.968940 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.968922 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.975997 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.975977 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.976233 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.976217 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.976280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976242 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.976280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976260 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.976504 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.976482 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.976554 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976512 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.976554 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976529 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.976752 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.976737 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.976794 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976757 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.976794 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.976772 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.977009 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.976989 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.977053 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977012 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.977053 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977024 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.977228 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.977207 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.977275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977234 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.977275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977250 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.977475 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.977438 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.977523 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977476 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.977523 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977490 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.977712 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:07:13.977687 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.977757 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977708 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.977757 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977726 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.977921 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977905 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.977975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.977921 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.978152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978132 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.978152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978151 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.978356 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978333 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.978449 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978357 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.978588 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978571 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.978638 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978589 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.978819 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978804 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.978819 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.978819 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.979062 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979045 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.979138 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979064 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.979287 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979269 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.979332 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979288 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.979470 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979439 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.979553 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979474 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.979686 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979672 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.979686 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979686 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.979920 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979902 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.979963 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.979921 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.980110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980094 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.980152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980111 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.980307 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980290 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.980352 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980308 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.980508 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980492 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.980609 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980510 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.980738 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980722 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.980796 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980739 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.980957 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980940 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.980995 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.980958 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.981184 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981164 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.981229 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981185 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.981415 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981398 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.981532 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981415 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.981677 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981661 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.981775 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981678 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.981903 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981875 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.981952 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.981904 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.982191 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982108 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.982191 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982132 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.982365 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982348 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.982412 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982366 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.982569 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982550 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.982569 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982567 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.982765 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982747 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.982805 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982769 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.982968 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982951 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.983006 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.982970 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.983177 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983161 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.983222 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983177 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.983347 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983332 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.983385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983347 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.983581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983566 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.983581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983581 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.983787 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983771 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.983787 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983788 2562 scope.go:117] "RemoveContainer" containerID="62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999" Jul 08 18:07:13.983989 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983975 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999"} err="failed to get container status \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": rpc error: code = NotFound desc = could not find container \"62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999\": container with ID starting with 62e018ae4ec149f975f81a0688f671c5a00e4e02b35aacc71dc823b0c48b5999 not found: ID does not exist" Jul 08 18:07:13.983989 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.983989 2562 scope.go:117] "RemoveContainer" containerID="1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8" Jul 08 18:07:13.984181 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984164 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8"} err="failed to get container status \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": rpc error: code = NotFound desc = could not find container \"1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8\": container with ID starting with 1bc47cc1a74cdb5c313e5a733154bdb9eb27e2bf7bdaa66ff3717aba540c0dc8 not found: ID does not exist" Jul 08 18:07:13.984216 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984181 2562 scope.go:117] "RemoveContainer" containerID="798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717" Jul 08 18:07:13.984377 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984352 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717"} err="failed to get container status \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": rpc error: code = NotFound desc = could not find container \"798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717\": container with ID starting with 798b1ee9ae9418fec245509435473f98894be68e86f0850f7f698105579a2717 not found: ID does not exist" Jul 08 18:07:13.984422 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984377 2562 scope.go:117] "RemoveContainer" containerID="1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b" Jul 08 18:07:13.984598 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984581 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b"} err="failed to get container status \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": rpc error: code = NotFound desc = could not find container \"1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b\": container with ID starting with 1ad9081ee0f20cb3ca0fe2d00f8460efd476c788c54dcc2c89e0a73c4933dc7b not found: ID does not exist" Jul 08 18:07:13.984644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984598 2562 scope.go:117] "RemoveContainer" containerID="9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403" Jul 08 18:07:13.984826 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984808 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403"} err="failed to get container status \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": rpc error: code = NotFound desc = could not find container \"9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403\": container with ID starting with 9a45fd93f035fbae1258b6b1dc50169c9339cedca9ef9cd7ade287573f581403 not found: ID does not exist" Jul 08 18:07:13.984907 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.984827 2562 scope.go:117] "RemoveContainer" containerID="1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd" Jul 08 18:07:13.985066 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.985049 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd"} err="failed to get container status \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": rpc error: code = NotFound desc = could not find container \"1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd\": container with ID starting with 1b34f77470824004bc6c7c031076762edb2ad13752fdb9933d7435ab51fb01cd not found: ID does not exist" Jul 08 18:07:13.985126 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.985068 2562 scope.go:117] "RemoveContainer" containerID="d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737" Jul 08 18:07:13.985300 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.985284 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737"} err="failed to get container status \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": rpc error: code = NotFound desc = could not find container \"d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737\": container with ID starting with d85742145b970789c6e32d3d727b2dcfd3ecdc8d40e30f98483183585e57f737 not found: ID does not exist" Jul 08 18:07:13.996313 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996295 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:13.996577 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996565 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-web" Jul 08 18:07:13.996618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996579 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-web" Jul 08 18:07:13.996618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996588 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy" Jul 08 18:07:13.996618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996594 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy" Jul 08 18:07:13.996618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996603 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="init-config-reloader" Jul 08 18:07:13.996618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996608 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="init-config-reloader" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996620 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="config-reloader" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996624 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="config-reloader" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996638 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="prometheus" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996643 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="prometheus" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996648 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="thanos-sidecar" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996653 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="thanos-sidecar" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996660 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-thanos" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996665 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-thanos" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996708 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="thanos-sidecar" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996717 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-thanos" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996722 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="prometheus" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996726 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy-web" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996732 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="config-reloader" Jul 08 18:07:13.996758 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.996737 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" containerName="kube-rbac-proxy" Jul 08 18:07:13.999996 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:13.999980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.008558 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008537 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 18:07:14.008650 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008592 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 18:07:14.008650 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008638 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:07:14.008772 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008756 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wx8zb\"" Jul 08 18:07:14.008816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 18:07:14.008864 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008822 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 18:07:14.008974 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008960 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 18:07:14.009048 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.008977 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 18:07:14.009048 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 18:07:14.009185 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009173 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 18:07:14.009393 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009379 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bg1h9av3lduh6\"" Jul 08 18:07:14.009546 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009533 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 18:07:14.009608 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009543 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 18:07:14.009608 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 18:07:14.009775 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.009761 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 18:07:14.014088 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.013867 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 18:07:14.018323 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.016572 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 18:07:14.028355 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.028333 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:14.079488 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079464 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079495 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-config-out\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079570 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q449g\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-kube-api-access-q449g\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-web-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079839 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.079886 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.080024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.080024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.080024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.080024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079954 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.080024 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.079969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181144 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181144 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181144 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181144 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-config-out\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q449g\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-kube-api-access-q449g\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-web-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.181790 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.182517 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.181846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.182517 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.182028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.182942 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.182631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.184502 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.184475 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.184606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.184536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.184606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.184568 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.184715 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.184680 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.185777 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.185140 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-config-out\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.185777 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.185382 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.185777 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.185673 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.185969 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.185795 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.186019 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.185979 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.186263 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.186221 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.186263 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.186238 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88dda251-d03d-4683-aa67-75eca4aeb926-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.187223 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.187198 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.187353 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.187330 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.187436 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.187418 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.187823 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.187800 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.188631 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.188612 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88dda251-d03d-4683-aa67-75eca4aeb926-web-config\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.192399 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.192376 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0" path="/var/lib/kubelet/pods/1b9c7e23-46ee-4fd6-92d3-d25cddf7ada0/volumes" Jul 08 18:07:14.194112 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.194094 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q449g\" (UniqueName: \"kubernetes.io/projected/88dda251-d03d-4683-aa67-75eca4aeb926-kube-api-access-q449g\") pod \"prometheus-k8s-0\" (UID: \"88dda251-d03d-4683-aa67-75eca4aeb926\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.309114 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.309082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:07:14.466759 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.466736 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 18:07:14.469168 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:07:14.469141 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88dda251_d03d_4683_aa67_75eca4aeb926.slice/crio-f4a7aadb16b259f6d96d67fd02c781158282daba4b7cd1d6b7b607d4ee590f00 WatchSource:0}: Error finding container f4a7aadb16b259f6d96d67fd02c781158282daba4b7cd1d6b7b607d4ee590f00: Status 404 returned error can't find the container with id f4a7aadb16b259f6d96d67fd02c781158282daba4b7cd1d6b7b607d4ee590f00 Jul 08 18:07:14.933179 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.933093 2562 generic.go:356] "Generic (PLEG): container finished" podID="88dda251-d03d-4683-aa67-75eca4aeb926" containerID="9c59f75df78cb60b4359f66741f826e440af9c456e3620aa546ff2ad168c9ba3" exitCode=0 Jul 08 18:07:14.933179 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.933147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerDied","Data":"9c59f75df78cb60b4359f66741f826e440af9c456e3620aa546ff2ad168c9ba3"} Jul 08 18:07:14.933363 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:14.933196 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"f4a7aadb16b259f6d96d67fd02c781158282daba4b7cd1d6b7b607d4ee590f00"} Jul 08 18:07:15.940568 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"21649a94ff1d429134cd57e3f04a1bcd927a1bbd5be3c5cc1f9dcd8e3055ef6f"} Jul 08 18:07:15.940924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"6723306b2b986ab55afe17d466ef02958714cdd44b94ce76f9c4845ab27efea1"} Jul 08 18:07:15.940924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"c40c22e562d45a80e9aaca180cb3ad5e47e0017439dc07f026ec2942729c21d4"} Jul 08 18:07:15.940924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"e321bfb0df77d66ee8562707a203eafc3bff3e01c7e58f0adf7e9abd5a6a7acf"} Jul 08 18:07:15.940924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940696 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"3dd50f05995d0530e235f2cf8a6bd9b87e60f6a1e15ddb2262d6a428f84191d6"} Jul 08 18:07:15.940924 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.940717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88dda251-d03d-4683-aa67-75eca4aeb926","Type":"ContainerStarted","Data":"31174541583f3ac90ec3bae6c46047d45a47e728a9ef527c9904e4a66d7d3da1"} Jul 08 18:07:15.985497 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:15.985424 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.966609617 podStartE2EDuration="2.985405505s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.795888ms" isStatefulPod=false podCreationTimestamp="2026-07-08 18:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:07:15.983857451 +0000 UTC m=+150.323301497" watchObservedRunningTime="2026-07-08 18:07:15.985405505 +0000 UTC m=+150.324849513" Jul 08 18:07:19.310015 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:07:19.309983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:08:14.309542 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:08:14.309505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:08:14.325625 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:08:14.325590 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:08:15.190805 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:08:15.190778 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 18:09:25.794381 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.794338 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4w8m8"] Jul 08 18:09:25.796324 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.796307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:25.799288 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.799269 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 18:09:25.799445 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.799426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7gw2p\"" Jul 08 18:09:25.800267 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.800253 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 18:09:25.810262 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.810244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4w8m8"] Jul 08 18:09:25.933068 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.933041 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-bound-sa-token\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:25.933223 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:25.933077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbj5s\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-kube-api-access-qbj5s\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.033938 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.033909 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-bound-sa-token\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.034078 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.033945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbj5s\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-kube-api-access-qbj5s\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.043023 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.042997 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-bound-sa-token\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.043142 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.043077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbj5s\" (UniqueName: \"kubernetes.io/projected/7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939-kube-api-access-qbj5s\") pod \"cert-manager-55b4d954d5-4w8m8\" (UID: \"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939\") " pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.113417 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.113369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4w8m8" Jul 08 18:09:26.249119 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.249099 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4w8m8"] Jul 08 18:09:26.251128 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:09:26.251098 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fe56ca7_b7b2_4f8d_ba4e_b9dbd3833939.slice/crio-26aa957de2a2d13af50bc3de41e361b9e2118e08d00d77b27ba8e39779c964a0 WatchSource:0}: Error finding container 26aa957de2a2d13af50bc3de41e361b9e2118e08d00d77b27ba8e39779c964a0: Status 404 returned error can't find the container with id 26aa957de2a2d13af50bc3de41e361b9e2118e08d00d77b27ba8e39779c964a0 Jul 08 18:09:26.435379 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:26.435307 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4w8m8" event={"ID":"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939","Type":"ContainerStarted","Data":"26aa957de2a2d13af50bc3de41e361b9e2118e08d00d77b27ba8e39779c964a0"} Jul 08 18:09:29.450580 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:29.450546 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4w8m8" event={"ID":"7fe56ca7-b7b2-4f8d-ba4e-b9dbd3833939","Type":"ContainerStarted","Data":"d18a736fe03cfe880dd6166603f752b1769f173cac5adfd422a6b8ff5526e2e3"} Jul 08 18:09:29.469777 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:29.469731 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4w8m8" podStartSLOduration=1.677932414 podStartE2EDuration="4.469715506s" totalImagesPullingTime="2.791783092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:09:29.468437928 +0000 UTC m=+283.807881938" watchObservedRunningTime="2026-07-08 18:09:29.469715506 +0000 UTC m=+283.809159514" Jul 08 18:09:46.034989 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:46.034962 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 18:09:52.336387 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.336356 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg"] Jul 08 18:09:52.347512 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.347497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.352755 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.352717 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f5v9n\"" Jul 08 18:09:52.352875 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.352759 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 18:09:52.352875 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.352832 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 18:09:52.353621 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.353602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:09:52.353732 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.353607 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 18:09:52.354141 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.354124 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 18:09:52.373736 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.373709 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx"] Jul 08 18:09:52.378386 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.378368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.385990 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.385973 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pxw5w\"" Jul 08 18:09:52.386839 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.386819 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 18:09:52.387156 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.387136 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 18:09:52.387261 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.387136 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 18:09:52.387261 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.387232 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 18:09:52.388151 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.388135 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg"] Jul 08 18:09:52.402341 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.402323 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx"] Jul 08 18:09:52.445800 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.445776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.445909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.445812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d151273-acd8-40fa-b1f3-2c016254ba4a-manager-config\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.445909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.445856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6b6g\" (UniqueName: \"kubernetes.io/projected/3d151273-acd8-40fa-b1f3-2c016254ba4a-kube-api-access-t6b6g\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.445909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.445900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-metrics-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.546744 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.546926 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d151273-acd8-40fa-b1f3-2c016254ba4a-manager-config\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.546926 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6b6g\" (UniqueName: \"kubernetes.io/projected/3d151273-acd8-40fa-b1f3-2c016254ba4a-kube-api-access-t6b6g\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.546926 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-metrics-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.546926 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-575sd\" (UniqueName: \"kubernetes.io/projected/b14fb214-15ee-4e81-bf01-19f15167940a-kube-api-access-575sd\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.546926 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.547152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.547152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.546966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.547152 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.547003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.547755 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.547731 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3d151273-acd8-40fa-b1f3-2c016254ba4a-manager-config\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.549403 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.549376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.549517 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.549425 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d151273-acd8-40fa-b1f3-2c016254ba4a-metrics-cert\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.610565 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.610518 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6b6g\" (UniqueName: \"kubernetes.io/projected/3d151273-acd8-40fa-b1f3-2c016254ba4a-kube-api-access-t6b6g\") pod \"lws-controller-manager-68bbd4cf59-dzgbg\" (UID: \"3d151273-acd8-40fa-b1f3-2c016254ba4a\") " pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.648178 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-575sd\" (UniqueName: \"kubernetes.io/projected/b14fb214-15ee-4e81-bf01-19f15167940a-kube-api-access-575sd\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.648272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.648272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.648272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.648437 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.648773 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.648745 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-manifests\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.650574 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.650547 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b14fb214-15ee-4e81-bf01-19f15167940a-tmp\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.650845 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.650824 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-webhook-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.651020 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.651006 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b14fb214-15ee-4e81-bf01-19f15167940a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.656940 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.656914 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:52.664812 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.664790 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-575sd\" (UniqueName: \"kubernetes.io/projected/b14fb214-15ee-4e81-bf01-19f15167940a-kube-api-access-575sd\") pod \"opendatahub-operator-controller-manager-859965ff4-d6hzx\" (UID: \"b14fb214-15ee-4e81-bf01-19f15167940a\") " pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.688093 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.688076 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:52.847512 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.847490 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg"] Jul 08 18:09:52.849652 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:09:52.849624 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d151273_acd8_40fa_b1f3_2c016254ba4a.slice/crio-b5c02ab572a64dfb73f1d731d3b74ab2137d96b93c9a4a0526f4544194f5fcb6 WatchSource:0}: Error finding container b5c02ab572a64dfb73f1d731d3b74ab2137d96b93c9a4a0526f4544194f5fcb6: Status 404 returned error can't find the container with id b5c02ab572a64dfb73f1d731d3b74ab2137d96b93c9a4a0526f4544194f5fcb6 Jul 08 18:09:52.851448 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.851434 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:09:52.854178 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:52.854148 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx"] Jul 08 18:09:52.856571 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:09:52.856545 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb14fb214_15ee_4e81_bf01_19f15167940a.slice/crio-aac963882de7da4a0a1133b90e36319dcd55fadeca08087187253347e6e6f0fd WatchSource:0}: Error finding container aac963882de7da4a0a1133b90e36319dcd55fadeca08087187253347e6e6f0fd: Status 404 returned error can't find the container with id aac963882de7da4a0a1133b90e36319dcd55fadeca08087187253347e6e6f0fd Jul 08 18:09:53.542283 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:53.542196 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" event={"ID":"b14fb214-15ee-4e81-bf01-19f15167940a","Type":"ContainerStarted","Data":"aac963882de7da4a0a1133b90e36319dcd55fadeca08087187253347e6e6f0fd"} Jul 08 18:09:53.544582 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:53.544552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" event={"ID":"3d151273-acd8-40fa-b1f3-2c016254ba4a","Type":"ContainerStarted","Data":"b5c02ab572a64dfb73f1d731d3b74ab2137d96b93c9a4a0526f4544194f5fcb6"} Jul 08 18:09:57.565485 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:57.565433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" event={"ID":"3d151273-acd8-40fa-b1f3-2c016254ba4a","Type":"ContainerStarted","Data":"9e6e25c95e9fd96c72f4d9720be6edef2d7605250bef23da522405d1794471f3"} Jul 08 18:09:57.565988 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:57.565517 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:09:57.567011 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:57.566989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" event={"ID":"b14fb214-15ee-4e81-bf01-19f15167940a","Type":"ContainerStarted","Data":"c63c36fbd8458a84f6348b3e1d638c6796f4e3f8550bdd1129d68dd0ae73744e"} Jul 08 18:09:57.593684 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:57.593639 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" podStartSLOduration=1.481671652 podStartE2EDuration="5.593623121s" totalImagesPullingTime="4.111951469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:09:57.589874984 +0000 UTC m=+311.929318992" watchObservedRunningTime="2026-07-08 18:09:57.593623121 +0000 UTC m=+311.933067129" Jul 08 18:09:58.572476 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:58.572424 2562 generic.go:356] "Generic (PLEG): container finished" podID="b14fb214-15ee-4e81-bf01-19f15167940a" containerID="c63c36fbd8458a84f6348b3e1d638c6796f4e3f8550bdd1129d68dd0ae73744e" exitCode=0 Jul 08 18:09:58.573559 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:58.573533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" event={"ID":"b14fb214-15ee-4e81-bf01-19f15167940a","Type":"ContainerDied","Data":"c63c36fbd8458a84f6348b3e1d638c6796f4e3f8550bdd1129d68dd0ae73744e"} Jul 08 18:09:59.578768 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:59.578734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" event={"ID":"b14fb214-15ee-4e81-bf01-19f15167940a","Type":"ContainerStarted","Data":"19406b29977ce5d12b6aa5e13a07a521640c2a65ffbcfe6387f04d88417d9c50"} Jul 08 18:09:59.579127 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:59.578852 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:09:59.615778 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:09:59.615734 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" podStartSLOduration=1.953146112 podStartE2EDuration="7.615720282s" totalImagesPullingTime="4.637067265s" totalInitContainerRuntime="1.025506905s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:09:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:09:59.614056518 +0000 UTC m=+313.953500517" watchObservedRunningTime="2026-07-08 18:09:59.615720282 +0000 UTC m=+313.955164290" Jul 08 18:10:08.578605 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:08.578569 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68bbd4cf59-dzgbg" Jul 08 18:10:10.584965 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:10.584934 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-859965ff4-d6hzx" Jul 08 18:10:14.229514 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.229482 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-9hjnd"] Jul 08 18:10:14.231636 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.231620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.234258 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.234236 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 18:10:14.234359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.234260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 18:10:14.234765 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.234748 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 18:10:14.235246 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.235230 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 18:10:14.235332 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.235236 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ln2lj\"" Jul 08 18:10:14.253182 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.253161 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-9hjnd"] Jul 08 18:10:14.328984 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.328959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tls-certs\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.329134 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.329074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6tms\" (UniqueName: \"kubernetes.io/projected/236bd738-f67c-4fac-8bff-c14ae5ad22dc-kube-api-access-d6tms\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.329134 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.329102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tmp\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.429782 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.429755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6tms\" (UniqueName: \"kubernetes.io/projected/236bd738-f67c-4fac-8bff-c14ae5ad22dc-kube-api-access-d6tms\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.429962 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.429787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tmp\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.429962 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.429812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tls-certs\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.432070 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.432048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tmp\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.432191 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.432171 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/236bd738-f67c-4fac-8bff-c14ae5ad22dc-tls-certs\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.458034 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.458010 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6tms\" (UniqueName: \"kubernetes.io/projected/236bd738-f67c-4fac-8bff-c14ae5ad22dc-kube-api-access-d6tms\") pod \"kube-auth-proxy-6784757fb-9hjnd\" (UID: \"236bd738-f67c-4fac-8bff-c14ae5ad22dc\") " pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.540628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.540554 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" Jul 08 18:10:14.715255 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:14.715233 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6784757fb-9hjnd"] Jul 08 18:10:14.717726 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:10:14.717701 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236bd738_f67c_4fac_8bff_c14ae5ad22dc.slice/crio-5b0272bf1fe3f4c612e4f182ecb5dde27ab200c0c39325be11f6e41ac7cfdc30 WatchSource:0}: Error finding container 5b0272bf1fe3f4c612e4f182ecb5dde27ab200c0c39325be11f6e41ac7cfdc30: Status 404 returned error can't find the container with id 5b0272bf1fe3f4c612e4f182ecb5dde27ab200c0c39325be11f6e41ac7cfdc30 Jul 08 18:10:15.647977 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:15.647945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" event={"ID":"236bd738-f67c-4fac-8bff-c14ae5ad22dc","Type":"ContainerStarted","Data":"5b0272bf1fe3f4c612e4f182ecb5dde27ab200c0c39325be11f6e41ac7cfdc30"} Jul 08 18:10:19.667016 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:19.666980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" event={"ID":"236bd738-f67c-4fac-8bff-c14ae5ad22dc","Type":"ContainerStarted","Data":"b2700d5917ab43e3c73451cac660a65cabcb242851b1e264e00a54499ac5a910"} Jul 08 18:10:19.699843 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:19.699792 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6784757fb-9hjnd" podStartSLOduration=0.908647484 podStartE2EDuration="5.699776786s" totalImagesPullingTime="4.791129302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:19.699013111 +0000 UTC m=+334.038457119" watchObservedRunningTime="2026-07-08 18:10:19.699776786 +0000 UTC m=+334.039220793" Jul 08 18:10:26.583330 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.583295 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9"] Jul 08 18:10:26.585538 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.585523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:26.590638 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.590608 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 18:10:26.590899 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.590884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lrjj4\"" Jul 08 18:10:26.614842 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.614817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9"] Jul 08 18:10:26.740289 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.740261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:26.740447 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.740310 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs2sj\" (UniqueName: \"kubernetes.io/projected/810e8946-94c1-46d4-b6f6-3f2500d7a501-kube-api-access-fs2sj\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:26.841581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.841507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs2sj\" (UniqueName: \"kubernetes.io/projected/810e8946-94c1-46d4-b6f6-3f2500d7a501-kube-api-access-fs2sj\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:26.841581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.841578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:26.841753 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:10:26.841685 2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 08 18:10:26.841753 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:10:26.841735 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert podName:810e8946-94c1-46d4-b6f6-3f2500d7a501 nodeName:}" failed. No retries permitted until 2026-07-08 18:10:27.341719289 +0000 UTC m=+341.681163275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert") pod "llmisvc-controller-manager-777c6bd7d5-286x9" (UID: "810e8946-94c1-46d4-b6f6-3f2500d7a501") : secret "llmisvc-webhook-server-cert" not found Jul 08 18:10:26.862727 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:26.862695 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs2sj\" (UniqueName: \"kubernetes.io/projected/810e8946-94c1-46d4-b6f6-3f2500d7a501-kube-api-access-fs2sj\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:27.344660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:27.344623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:27.347088 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:27.347068 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/810e8946-94c1-46d4-b6f6-3f2500d7a501-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-286x9\" (UID: \"810e8946-94c1-46d4-b6f6-3f2500d7a501\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:27.495014 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:27.494978 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:27.631332 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:27.631255 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9"] Jul 08 18:10:27.635092 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:10:27.635067 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod810e8946_94c1_46d4_b6f6_3f2500d7a501.slice/crio-a4880b413d03aca0a7e9ab14964b5626d25228cfc3d64201dfff0d293603bc00 WatchSource:0}: Error finding container a4880b413d03aca0a7e9ab14964b5626d25228cfc3d64201dfff0d293603bc00: Status 404 returned error can't find the container with id a4880b413d03aca0a7e9ab14964b5626d25228cfc3d64201dfff0d293603bc00 Jul 08 18:10:27.702091 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:27.702063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" event={"ID":"810e8946-94c1-46d4-b6f6-3f2500d7a501","Type":"ContainerStarted","Data":"a4880b413d03aca0a7e9ab14964b5626d25228cfc3d64201dfff0d293603bc00"} Jul 08 18:10:32.725900 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:32.725864 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" event={"ID":"810e8946-94c1-46d4-b6f6-3f2500d7a501","Type":"ContainerStarted","Data":"16ad5470954f40428df137334a41277d74397e5f54de5c25b7da9eb0e90aa158"} Jul 08 18:10:32.726306 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:32.725985 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:10:32.754892 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:10:32.754843 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" podStartSLOduration=2.454233937 podStartE2EDuration="6.754825685s" totalImagesPullingTime="4.300591748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:10:32.754670148 +0000 UTC m=+347.094114156" watchObservedRunningTime="2026-07-08 18:10:32.754825685 +0000 UTC m=+347.094269693" Jul 08 18:11:03.732407 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:11:03.732332 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-286x9" Jul 08 18:12:00.248690 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.248657 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s"] Jul 08 18:12:00.250990 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.250975 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.253703 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.253685 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6gbkt\"" Jul 08 18:12:00.253808 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.253689 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 18:12:00.254401 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.254389 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 18:12:00.255082 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.255070 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 18:12:00.255321 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.255310 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 18:12:00.260573 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.260553 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s"] Jul 08 18:12:00.374246 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.374219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb31782c-5454-4762-8459-6afd4600785c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.374370 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.374252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczz8\" (UniqueName: \"kubernetes.io/projected/cb31782c-5454-4762-8459-6afd4600785c-kube-api-access-rczz8\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.374370 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.374338 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.474748 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.474706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.474896 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.474763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb31782c-5454-4762-8459-6afd4600785c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.474896 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:12:00.474846 2562 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 18:12:00.474896 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.474885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczz8\" (UniqueName: \"kubernetes.io/projected/cb31782c-5454-4762-8459-6afd4600785c-kube-api-access-rczz8\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.475060 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:12:00.474910 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert podName:cb31782c-5454-4762-8459-6afd4600785c nodeName:}" failed. No retries permitted until 2026-07-08 18:12:00.97489316 +0000 UTC m=+435.314337145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-pmn7s" (UID: "cb31782c-5454-4762-8459-6afd4600785c") : secret "plugin-serving-cert" not found Jul 08 18:12:00.475311 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.475294 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb31782c-5454-4762-8459-6afd4600785c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.493692 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.493672 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczz8\" (UniqueName: \"kubernetes.io/projected/cb31782c-5454-4762-8459-6afd4600785c-kube-api-access-rczz8\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.979290 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.979261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:00.981707 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:00.981688 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb31782c-5454-4762-8459-6afd4600785c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-pmn7s\" (UID: \"cb31782c-5454-4762-8459-6afd4600785c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:01.160303 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:01.160267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" Jul 08 18:12:01.286678 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:01.286637 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s"] Jul 08 18:12:01.289508 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:12:01.289483 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb31782c_5454_4762_8459_6afd4600785c.slice/crio-b47280e35235f4e2866fbb771b84b11cca0f4efa54066a007f7f7750e1468e45 WatchSource:0}: Error finding container b47280e35235f4e2866fbb771b84b11cca0f4efa54066a007f7f7750e1468e45: Status 404 returned error can't find the container with id b47280e35235f4e2866fbb771b84b11cca0f4efa54066a007f7f7750e1468e45 Jul 08 18:12:02.117692 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:02.117662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" event={"ID":"cb31782c-5454-4762-8459-6afd4600785c","Type":"ContainerStarted","Data":"b47280e35235f4e2866fbb771b84b11cca0f4efa54066a007f7f7750e1468e45"} Jul 08 18:12:27.239942 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:27.239910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" event={"ID":"cb31782c-5454-4762-8459-6afd4600785c","Type":"ContainerStarted","Data":"f570774c034fd0849f637f579da779604f4a23561c9638beac243092bb138178"} Jul 08 18:12:27.267964 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:27.267910 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-pmn7s" podStartSLOduration=1.952250088 podStartE2EDuration="27.267893934s" totalImagesPullingTime="25.315643846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:27.265735924 +0000 UTC m=+461.605179948" watchObservedRunningTime="2026-07-08 18:12:27.267893934 +0000 UTC m=+461.607337942" Jul 08 18:12:53.430807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.430773 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:12:53.618199 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.618168 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:12:53.618199 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.618207 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:12:53.618396 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.618280 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.622351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.622335 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 18:12:53.631733 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.631711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db112ac4-1357-49e9-a228-df2a620701ee-config-file\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.631812 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.631779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktzfr\" (UniqueName: \"kubernetes.io/projected/db112ac4-1357-49e9-a228-df2a620701ee-kube-api-access-ktzfr\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.732287 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.732260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktzfr\" (UniqueName: \"kubernetes.io/projected/db112ac4-1357-49e9-a228-df2a620701ee-kube-api-access-ktzfr\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.732415 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.732309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db112ac4-1357-49e9-a228-df2a620701ee-config-file\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.732863 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.732846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db112ac4-1357-49e9-a228-df2a620701ee-config-file\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.745101 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.745079 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktzfr\" (UniqueName: \"kubernetes.io/projected/db112ac4-1357-49e9-a228-df2a620701ee-kube-api-access-ktzfr\") pod \"limitador-limitador-54d6d988d5-vrtf8\" (UID: \"db112ac4-1357-49e9-a228-df2a620701ee\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:53.927972 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:53.927937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:54.138679 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.138652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:12:54.141207 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:12:54.141183 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb112ac4_1357_49e9_a228_df2a620701ee.slice/crio-00bf420f60ca17c916f0d7404773efd60c6d6bc8856fe42ca1e715cc2efd99bd WatchSource:0}: Error finding container 00bf420f60ca17c916f0d7404773efd60c6d6bc8856fe42ca1e715cc2efd99bd: Status 404 returned error can't find the container with id 00bf420f60ca17c916f0d7404773efd60c6d6bc8856fe42ca1e715cc2efd99bd Jul 08 18:12:54.156267 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.156245 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:12:54.192976 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.192950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:12:54.196234 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.196212 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7cf7r\"" Jul 08 18:12:54.199882 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.199861 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:12:54.236775 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.236750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx45x\" (UniqueName: \"kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x\") pod \"authorino-75d556bd77-p6l8r\" (UID: \"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8\") " pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:12:54.337959 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.337889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx45x\" (UniqueName: \"kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x\") pod \"authorino-75d556bd77-p6l8r\" (UID: \"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8\") " pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:12:54.354028 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.353995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx45x\" (UniqueName: \"kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x\") pod \"authorino-75d556bd77-p6l8r\" (UID: \"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8\") " pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:12:54.357223 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.357198 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" event={"ID":"db112ac4-1357-49e9-a228-df2a620701ee","Type":"ContainerStarted","Data":"00bf420f60ca17c916f0d7404773efd60c6d6bc8856fe42ca1e715cc2efd99bd"} Jul 08 18:12:54.503559 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.503526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:12:54.687247 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:54.687183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:12:54.692195 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:12:54.692165 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbe69e8c_9cc6_44e3_8c27_e1e2bee30ab8.slice/crio-5b6b746291fcaa25e3ac2167f6b22c34c91798265727623968b7286b485f552c WatchSource:0}: Error finding container 5b6b746291fcaa25e3ac2167f6b22c34c91798265727623968b7286b485f552c: Status 404 returned error can't find the container with id 5b6b746291fcaa25e3ac2167f6b22c34c91798265727623968b7286b485f552c Jul 08 18:12:55.362829 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:55.362796 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6l8r" event={"ID":"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8","Type":"ContainerStarted","Data":"5b6b746291fcaa25e3ac2167f6b22c34c91798265727623968b7286b485f552c"} Jul 08 18:12:57.376627 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:57.376537 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" event={"ID":"db112ac4-1357-49e9-a228-df2a620701ee","Type":"ContainerStarted","Data":"287cf29311a5477b89aff1d69bad306fb0bcff6b2ff1f727fe1508a0c4fcd21c"} Jul 08 18:12:57.377069 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:57.376631 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:12:57.400208 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:12:57.400146 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" podStartSLOduration=1.594707012 podStartE2EDuration="4.400127126s" totalImagesPullingTime="2.805420114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:12:57.398174913 +0000 UTC m=+491.737618921" watchObservedRunningTime="2026-07-08 18:12:57.400127126 +0000 UTC m=+491.739571138" Jul 08 18:13:00.392606 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:00.392571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6l8r" event={"ID":"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8","Type":"ContainerStarted","Data":"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e"} Jul 08 18:13:00.415066 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:00.415008 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-p6l8r" podStartSLOduration=1.670294712 podStartE2EDuration="6.414992328s" totalImagesPullingTime="4.744697616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:13:00.413543029 +0000 UTC m=+494.752987037" watchObservedRunningTime="2026-07-08 18:13:00.414992328 +0000 UTC m=+494.754436336" Jul 08 18:13:08.383027 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:08.382993 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-vrtf8" Jul 08 18:13:27.461612 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.461583 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:13:27.462095 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.461750 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-p6l8r" podUID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" containerName="authorino" containerID="cri-o://6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e" gracePeriod=30 Jul 08 18:13:27.499770 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:13:27.499743 2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbe69e8c_9cc6_44e3_8c27_e1e2bee30ab8.slice/crio-6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e.scope\": RecentStats: unable to find data in memory cache]" Jul 08 18:13:27.707745 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.707723 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:13:27.825671 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.825592 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx45x\" (UniqueName: \"kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x\") pod \"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8\" (UID: \"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8\") " Jul 08 18:13:27.827822 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.827791 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x" (OuterVolumeSpecName: "kube-api-access-zx45x") pod "cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" (UID: "cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8"). InnerVolumeSpecName "kube-api-access-zx45x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:13:27.926389 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:27.926356 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx45x\" (UniqueName: \"kubernetes.io/projected/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8-kube-api-access-zx45x\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:13:28.520445 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.520364 2562 generic.go:356] "Generic (PLEG): container finished" podID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" containerID="6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e" exitCode=0 Jul 08 18:13:28.520445 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.520407 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-p6l8r" Jul 08 18:13:28.520445 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.520435 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6l8r" event={"ID":"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8","Type":"ContainerDied","Data":"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e"} Jul 08 18:13:28.520911 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.520481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-p6l8r" event={"ID":"cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8","Type":"ContainerDied","Data":"5b6b746291fcaa25e3ac2167f6b22c34c91798265727623968b7286b485f552c"} Jul 08 18:13:28.520911 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.520496 2562 scope.go:117] "RemoveContainer" containerID="6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e" Jul 08 18:13:28.528605 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.528588 2562 scope.go:117] "RemoveContainer" containerID="6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e" Jul 08 18:13:28.528871 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:13:28.528847 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e\": container with ID starting with 6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e not found: ID does not exist" containerID="6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e" Jul 08 18:13:28.528909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.528882 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e"} err="failed to get container status \"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e\": rpc error: code = NotFound desc = could not find container \"6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e\": container with ID starting with 6effff0953ee9fc09085be414735ea236edde17e06f9e64af41beecea3ca2f4e not found: ID does not exist" Jul 08 18:13:28.552734 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.552713 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:13:28.559253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:28.559232 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-p6l8r"] Jul 08 18:13:30.191851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:13:30.190191 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" path="/var/lib/kubelet/pods/cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8/volumes" Jul 08 18:14:09.022902 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:09.022871 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:14:27.210599 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:27.210564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:14:41.365591 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.365559 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2"] Jul 08 18:14:41.366035 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.365904 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" containerName="authorino" Jul 08 18:14:41.366035 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.365914 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" containerName="authorino" Jul 08 18:14:41.366035 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.365981 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbe69e8c-9cc6-44e3-8c27-e1e2bee30ab8" containerName="authorino" Jul 08 18:14:41.368975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.368961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.373762 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.373741 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 18:14:41.373879 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.373744 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 18:14:41.374395 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.374377 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 18:14:41.374510 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.374383 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-l6gpw\"" Jul 08 18:14:41.392923 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.392904 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2"] Jul 08 18:14:41.463963 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.463939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.464079 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.463981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2r6\" (UniqueName: \"kubernetes.io/projected/522f575e-d081-41e1-aec9-f961540cd199-kube-api-access-gb2r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.464079 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.464001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.464079 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.464017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522f575e-d081-41e1-aec9-f961540cd199-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.464192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.464088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.464192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.464157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.564951 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.564922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2r6\" (UniqueName: \"kubernetes.io/projected/522f575e-d081-41e1-aec9-f961540cd199-kube-api-access-gb2r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.564959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.564977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522f575e-d081-41e1-aec9-f961540cd199-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565103 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565367 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565502 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565482 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.565561 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.565506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.567403 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.567386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522f575e-d081-41e1-aec9-f961540cd199-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.567578 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.567563 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522f575e-d081-41e1-aec9-f961540cd199-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.574975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.574956 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2r6\" (UniqueName: \"kubernetes.io/projected/522f575e-d081-41e1-aec9-f961540cd199-kube-api-access-gb2r6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bvsk2\" (UID: \"522f575e-d081-41e1-aec9-f961540cd199\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.679104 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.679047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:41.827674 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:41.827650 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2"] Jul 08 18:14:41.830432 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:14:41.830408 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522f575e_d081_41e1_aec9_f961540cd199.slice/crio-950d4454f9f01585ef51994694ed9e904e801c8553eeffd45b16adb340f1e5cb WatchSource:0}: Error finding container 950d4454f9f01585ef51994694ed9e904e801c8553eeffd45b16adb340f1e5cb: Status 404 returned error can't find the container with id 950d4454f9f01585ef51994694ed9e904e801c8553eeffd45b16adb340f1e5cb Jul 08 18:14:42.383241 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:42.383205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:14:42.841368 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:42.841339 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" event={"ID":"522f575e-d081-41e1-aec9-f961540cd199","Type":"ContainerStarted","Data":"950d4454f9f01585ef51994694ed9e904e801c8553eeffd45b16adb340f1e5cb"} Jul 08 18:14:43.847009 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:43.846974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" event={"ID":"522f575e-d081-41e1-aec9-f961540cd199","Type":"ContainerStarted","Data":"e600a446ab159dc3ef6b284e7872b537811abc93be70638849985c676ba964e2"} Jul 08 18:14:43.887341 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:43.882890 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" podStartSLOduration=1.817252153 podStartE2EDuration="2.882870436s" totalImagesPullingTime="1.065618283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:14:43.88113734 +0000 UTC m=+598.220581348" watchObservedRunningTime="2026-07-08 18:14:43.882870436 +0000 UTC m=+598.222314446" Jul 08 18:14:51.680203 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:51.680152 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:51.680646 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:51.680223 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:51.694258 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:51.694237 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:51.892334 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:51.892294 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bvsk2" Jul 08 18:14:53.098272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:53.098237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:14:57.632134 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:14:57.632102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:15:00.188954 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.188926 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:15:00.193639 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.193620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:15:00.198038 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.198018 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 18:15:00.198145 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.198022 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 18:15:00.198879 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.198862 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m6n45\"" Jul 08 18:15:00.208330 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.208304 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:15:00.327855 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.327818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dscpt\" (UniqueName: \"kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt\") pod \"maas-api-key-cleanup-29725575-686zb\" (UID: \"831e8d81-9137-4354-8d8c-0294464f8eba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:15:00.429683 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.429637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dscpt\" (UniqueName: \"kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt\") pod \"maas-api-key-cleanup-29725575-686zb\" (UID: \"831e8d81-9137-4354-8d8c-0294464f8eba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:15:00.444200 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.444127 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dscpt\" (UniqueName: \"kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt\") pod \"maas-api-key-cleanup-29725575-686zb\" (UID: \"831e8d81-9137-4354-8d8c-0294464f8eba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:15:00.503253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.503211 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:15:00.635724 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.635693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:15:00.639528 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:15:00.639497 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod831e8d81_9137_4354_8d8c_0294464f8eba.slice/crio-135821d6139a51e065e98d5c3dfd485bb86c4dbb5596e4312a80f2321390df50 WatchSource:0}: Error finding container 135821d6139a51e065e98d5c3dfd485bb86c4dbb5596e4312a80f2321390df50: Status 404 returned error can't find the container with id 135821d6139a51e065e98d5c3dfd485bb86c4dbb5596e4312a80f2321390df50 Jul 08 18:15:00.641970 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.641955 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:15:00.927231 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:00.927157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" event={"ID":"831e8d81-9137-4354-8d8c-0294464f8eba","Type":"ContainerStarted","Data":"135821d6139a51e065e98d5c3dfd485bb86c4dbb5596e4312a80f2321390df50"} Jul 08 18:15:03.943107 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:03.943078 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" event={"ID":"831e8d81-9137-4354-8d8c-0294464f8eba","Type":"ContainerStarted","Data":"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212"} Jul 08 18:15:03.974483 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:03.974431 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" podStartSLOduration=1.3510446649999999 podStartE2EDuration="3.974414215s" totalImagesPullingTime="2.62336955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:15:03.973127493 +0000 UTC m=+618.312571500" watchObservedRunningTime="2026-07-08 18:15:03.974414215 +0000 UTC m=+618.313858226" Jul 08 18:15:06.236551 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.236515 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d"] Jul 08 18:15:06.240236 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.240209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.248005 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.247984 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 18:15:06.257649 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.257625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d"] Jul 08 18:15:06.379633 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.379633 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngst5\" (UniqueName: \"kubernetes.io/projected/36d57ea2-4417-46fd-9431-6f2a221f4a55-kube-api-access-ngst5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.379795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.379795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.379795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d57ea2-4417-46fd-9431-6f2a221f4a55-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.379795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.379760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.480828 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.480797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d57ea2-4417-46fd-9431-6f2a221f4a55-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.480921 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.480835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.480921 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.480878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481003 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.480930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngst5\" (UniqueName: \"kubernetes.io/projected/36d57ea2-4417-46fd-9431-6f2a221f4a55-kube-api-access-ngst5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481003 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.480967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481097 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.481016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481379 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.481341 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481379 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.481366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.481553 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.481393 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.483317 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.483287 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36d57ea2-4417-46fd-9431-6f2a221f4a55-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.483608 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.483591 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36d57ea2-4417-46fd-9431-6f2a221f4a55-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.495478 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.495410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngst5\" (UniqueName: \"kubernetes.io/projected/36d57ea2-4417-46fd-9431-6f2a221f4a55-kube-api-access-ngst5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4rt2d\" (UID: \"36d57ea2-4417-46fd-9431-6f2a221f4a55\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.550817 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.550794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:06.697730 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.697693 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d"] Jul 08 18:15:06.700225 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:15:06.700198 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d57ea2_4417_46fd_9431_6f2a221f4a55.slice/crio-6030208b98fa5a807c940dbdc098df46cc294923ba4aeb52e17655090f7a94ee WatchSource:0}: Error finding container 6030208b98fa5a807c940dbdc098df46cc294923ba4aeb52e17655090f7a94ee: Status 404 returned error can't find the container with id 6030208b98fa5a807c940dbdc098df46cc294923ba4aeb52e17655090f7a94ee Jul 08 18:15:06.964724 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:06.964690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" event={"ID":"36d57ea2-4417-46fd-9431-6f2a221f4a55","Type":"ContainerStarted","Data":"6030208b98fa5a807c940dbdc098df46cc294923ba4aeb52e17655090f7a94ee"} Jul 08 18:15:07.970739 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:07.970702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" event={"ID":"36d57ea2-4417-46fd-9431-6f2a221f4a55","Type":"ContainerStarted","Data":"926a051abd23efd68c4b38ecde225ba1e5cc136ae7c50b6017fbf2d7382f52a4"} Jul 08 18:15:08.010241 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:08.010194 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" podStartSLOduration=1.742758581 podStartE2EDuration="2.010177377s" totalImagesPullingTime="267.418796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:15:08.009233252 +0000 UTC m=+622.348677260" watchObservedRunningTime="2026-07-08 18:15:08.010177377 +0000 UTC m=+622.349621384" Jul 08 18:15:08.427063 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:08.426983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:15:14.118603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.118575 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k"] Jul 08 18:15:14.122075 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.122056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.125696 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.125677 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 18:15:14.139677 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.139652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k"] Jul 08 18:15:14.142382 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.142538 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.142538 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.142637 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.142637 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.142637 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.142620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swsmg\" (UniqueName: \"kubernetes.io/projected/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-kube-api-access-swsmg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243536 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243702 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243702 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swsmg\" (UniqueName: \"kubernetes.io/projected/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-kube-api-access-swsmg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243702 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243702 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.243910 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.244006 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.243983 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.244059 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.244030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.244095 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.244076 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.246121 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.246101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.246253 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.246236 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.259335 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.259310 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swsmg\" (UniqueName: \"kubernetes.io/projected/f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac-kube-api-access-swsmg\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k\" (UID: \"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.432860 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.432770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:14.588604 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:14.588574 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k"] Jul 08 18:15:14.592795 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:15:14.592768 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3d708e4_d4b4_429f_bcbc_9d57b3ea47ac.slice/crio-762c979556bbc1a74074b4ae68106ebf791ac49e21c87cf351e9e9f06372a635 WatchSource:0}: Error finding container 762c979556bbc1a74074b4ae68106ebf791ac49e21c87cf351e9e9f06372a635: Status 404 returned error can't find the container with id 762c979556bbc1a74074b4ae68106ebf791ac49e21c87cf351e9e9f06372a635 Jul 08 18:15:15.006366 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:15.006333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" event={"ID":"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac","Type":"ContainerStarted","Data":"48b407f91c1f977b068b00ed7cc01959d850151b1d44870740a8c19496fcea0a"} Jul 08 18:15:15.006537 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:15.006374 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" event={"ID":"f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac","Type":"ContainerStarted","Data":"762c979556bbc1a74074b4ae68106ebf791ac49e21c87cf351e9e9f06372a635"} Jul 08 18:15:15.038512 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:15.038448 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" podStartSLOduration=0.837102476 podStartE2EDuration="1.03843285s" totalImagesPullingTime="201.330374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:15:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:15:15.034041257 +0000 UTC m=+629.373485264" watchObservedRunningTime="2026-07-08 18:15:15.03843285 +0000 UTC m=+629.377876857" Jul 08 18:15:16.214700 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:16.214662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:15:16.551518 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:16.551429 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:16.551518 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:16.551495 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:16.563618 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:16.563596 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:17.026471 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:17.026428 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4rt2d" Jul 08 18:15:24.433580 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:24.433544 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:24.433580 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:24.433592 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:24.446133 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:24.446108 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:15:25.065836 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:15:25.065807 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k" Jul 08 18:16:00.013514 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:00.013480 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:16:00.014000 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:00.013677 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" podUID="831e8d81-9137-4354-8d8c-0294464f8eba" containerName="cleanup" containerID="cri-o://1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212" gracePeriod=30 Jul 08 18:16:30.190715 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.190693 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:16:30.307789 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.307762 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dscpt\" (UniqueName: \"kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt\") pod \"831e8d81-9137-4354-8d8c-0294464f8eba\" (UID: \"831e8d81-9137-4354-8d8c-0294464f8eba\") " Jul 08 18:16:30.310016 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.309990 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt" (OuterVolumeSpecName: "kube-api-access-dscpt") pod "831e8d81-9137-4354-8d8c-0294464f8eba" (UID: "831e8d81-9137-4354-8d8c-0294464f8eba"). InnerVolumeSpecName "kube-api-access-dscpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:16:30.338154 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.338128 2562 generic.go:356] "Generic (PLEG): container finished" podID="831e8d81-9137-4354-8d8c-0294464f8eba" containerID="1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212" exitCode=137 Jul 08 18:16:30.338250 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.338170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" event={"ID":"831e8d81-9137-4354-8d8c-0294464f8eba","Type":"ContainerDied","Data":"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212"} Jul 08 18:16:30.338250 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.338173 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" Jul 08 18:16:30.338250 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.338209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb" event={"ID":"831e8d81-9137-4354-8d8c-0294464f8eba","Type":"ContainerDied","Data":"135821d6139a51e065e98d5c3dfd485bb86c4dbb5596e4312a80f2321390df50"} Jul 08 18:16:30.338250 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.338230 2562 scope.go:117] "RemoveContainer" containerID="1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212" Jul 08 18:16:30.347300 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.347283 2562 scope.go:117] "RemoveContainer" containerID="1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212" Jul 08 18:16:30.347571 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:16:30.347553 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212\": container with ID starting with 1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212 not found: ID does not exist" containerID="1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212" Jul 08 18:16:30.347611 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.347581 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212"} err="failed to get container status \"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212\": rpc error: code = NotFound desc = could not find container \"1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212\": container with ID starting with 1427bc3a859c97df8f7c6cfb78368693611f709dce059d013e3be7bfe3d79212 not found: ID does not exist" Jul 08 18:16:30.359599 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.359575 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:16:30.368237 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.368216 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725575-686zb"] Jul 08 18:16:30.408519 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:30.408494 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dscpt\" (UniqueName: \"kubernetes.io/projected/831e8d81-9137-4354-8d8c-0294464f8eba-kube-api-access-dscpt\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:16:32.196191 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:32.196162 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="831e8d81-9137-4354-8d8c-0294464f8eba" path="/var/lib/kubelet/pods/831e8d81-9137-4354-8d8c-0294464f8eba/volumes" Jul 08 18:16:50.289076 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:16:50.289036 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:17:00.303744 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:17:00.303715 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:17:10.588071 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:17:10.588030 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:17:20.990774 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:17:20.990740 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:17:30.377189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:17:30.377161 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:17:59.218254 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:17:59.218219 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:18:10.079889 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:18:10.079810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:18:44.880704 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:18:44.880669 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:18:54.680986 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:18:54.680951 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:02.176795 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:02.176759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:12.989837 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:12.989791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:23.487670 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:23.487637 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:33.183643 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:33.183612 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:43.680300 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:43.680223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:19:53.280318 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:19:53.280280 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:20:15.382848 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:20:15.382815 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:20:26.986972 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:20:26.986940 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:20:51.084272 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:20:51.084243 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:01.284566 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:01.284531 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:16.089750 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:16.089673 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:24.786249 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:24.786210 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:34.283589 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:34.283556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:44.391353 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:44.391322 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:21:53.295380 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:21:53.295345 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:23:43.384787 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:23:43.384751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:23:54.772068 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:23:54.772031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:05.485045 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:05.485012 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:14.074237 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:14.074154 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:24.371986 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:24.371955 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:32.180968 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:32.180931 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:41.378029 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:41.377991 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:24:52.872970 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:24:52.872937 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:25:01.981550 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:25:01.981511 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:25:12.572468 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:25:12.572420 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:25:19.885254 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:25:19.885220 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:25:31.779200 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:25:31.779164 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:25:54.078156 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:25:54.078120 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:02.985487 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:02.985440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:13.974189 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:13.974155 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:25.780313 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:25.780277 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:34.575496 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:34.575440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:42.281851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:42.281818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:26:51.477097 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:26:51.477059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:00.603245 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:00.603210 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:08.574268 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:08.574192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:19.478975 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:19.478938 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:28.477688 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:28.477655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:42.384420 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:42.384385 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:27:51.386766 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:27:51.386730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:01.077566 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:01.077528 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:11.280102 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:11.280066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:22.973546 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:22.973512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:31.084403 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:31.084370 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:43.578271 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:43.578185 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:28:54.572382 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:28:54.572339 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:29:03.874616 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:29:03.874578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:29:16.886839 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:29:16.886803 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:29:28.478537 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:29:28.478498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:29:52.282051 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:29:52.282018 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:30:00.880257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:30:00.880223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:30:24.173474 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:30:24.173384 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:30:32.777937 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:30:32.777902 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:31:56.567996 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:31:56.567910 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:32:05.774558 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:32:05.774523 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:32:17.377735 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:32:17.374537 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:32:28.695380 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:32:28.695324 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:33:01.873820 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:01.873787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:33:13.951106 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.951028 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:33:13.951477 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.951423 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="831e8d81-9137-4354-8d8c-0294464f8eba" containerName="cleanup" Jul 08 18:33:13.951477 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.951434 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="831e8d81-9137-4354-8d8c-0294464f8eba" containerName="cleanup" Jul 08 18:33:13.951561 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.951512 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="831e8d81-9137-4354-8d8c-0294464f8eba" containerName="cleanup" Jul 08 18:33:13.954581 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.954565 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.957947 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.957922 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ef261af\"/\"e2e-lifecycle-model-3ef261af-kserve-self-signed-certs\"" Jul 08 18:33:13.958066 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.958010 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ef261af\"/\"kube-root-ca.crt\"" Jul 08 18:33:13.958066 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.958040 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ef261af\"/\"default-dockercfg-6pjm2\"" Jul 08 18:33:13.958173 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.958092 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ef261af\"/\"openshift-service-ca.crt\"" Jul 08 18:33:13.960208 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.960301 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.960301 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.960391 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960366 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw4x8\" (UniqueName: \"kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.960438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.960504 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.960478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:13.969764 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:13.969738 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:33:14.061283 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061283 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw4x8\" (UniqueName: \"kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061767 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.061889 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.061851 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.063641 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.063623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.063880 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.063865 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.070614 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.070593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw4x8\" (UniqueName: \"kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8\") pod \"e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.263296 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.263195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:14.397749 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.397719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:33:14.401133 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:33:14.401108 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d46c367_b3ba_419e_9867_91648c413314.slice/crio-34d0867e4cbd599f16767d681b864c1f398c417eaeb01a294566094d953a2dc3 WatchSource:0}: Error finding container 34d0867e4cbd599f16767d681b864c1f398c417eaeb01a294566094d953a2dc3: Status 404 returned error can't find the container with id 34d0867e4cbd599f16767d681b864c1f398c417eaeb01a294566094d953a2dc3 Jul 08 18:33:14.832708 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.832673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" event={"ID":"2d46c367-b3ba-419e-9867-91648c413314","Type":"ContainerStarted","Data":"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779"} Jul 08 18:33:14.832881 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:14.832717 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" event={"ID":"2d46c367-b3ba-419e-9867-91648c413314","Type":"ContainerStarted","Data":"34d0867e4cbd599f16767d681b864c1f398c417eaeb01a294566094d953a2dc3"} Jul 08 18:33:17.281799 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:17.281745 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" podStartSLOduration=4.28172661 podStartE2EDuration="4.28172661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:33:14.870687346 +0000 UTC m=+1709.210131355" watchObservedRunningTime="2026-07-08 18:33:17.28172661 +0000 UTC m=+1711.621170617" Jul 08 18:33:17.282185 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:17.282132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:33:24.263513 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:24.263473 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:24.263998 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:24.263533 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:24.275689 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:24.275666 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:24.890202 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:24.890176 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:29.974192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:29.974154 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:33:42.871298 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:42.871261 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:33:43.291359 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:43.291321 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:33:58.312546 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.312489 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" podUID="2d46c367-b3ba-419e-9867-91648c413314" containerName="main" containerID="cri-o://3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779" gracePeriod=45 Jul 08 18:33:58.555534 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.555510 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:58.652379 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652303 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652379 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652407 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652508 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw4x8\" (UniqueName: \"kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652539 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs\") pod \"2d46c367-b3ba-419e-9867-91648c413314\" (UID: \"2d46c367-b3ba-419e-9867-91648c413314\") " Jul 08 18:33:58.652628 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652573 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home" (OuterVolumeSpecName: "home") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:33:58.652820 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652665 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:33:58.652820 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652679 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:33:58.652893 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652855 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-model-cache\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:58.652893 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652869 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-tmp-dir\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:58.652893 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.652879 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-home\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:58.654791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.654765 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:33:58.654791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.654775 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm" (OuterVolumeSpecName: "dshm") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:33:58.654906 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.654810 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8" (OuterVolumeSpecName: "kube-api-access-lw4x8") pod "2d46c367-b3ba-419e-9867-91648c413314" (UID: "2d46c367-b3ba-419e-9867-91648c413314"). InnerVolumeSpecName "kube-api-access-lw4x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:33:58.753516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.753490 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d46c367-b3ba-419e-9867-91648c413314-dshm\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:58.753516 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.753512 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw4x8\" (UniqueName: \"kubernetes.io/projected/2d46c367-b3ba-419e-9867-91648c413314-kube-api-access-lw4x8\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:58.753665 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:58.753523 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d46c367-b3ba-419e-9867-91648c413314-tls-certs\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:33:59.032604 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.032570 2562 generic.go:356] "Generic (PLEG): container finished" podID="2d46c367-b3ba-419e-9867-91648c413314" containerID="3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779" exitCode=0 Jul 08 18:33:59.032791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.032622 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" Jul 08 18:33:59.032791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.032644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" event={"ID":"2d46c367-b3ba-419e-9867-91648c413314","Type":"ContainerDied","Data":"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779"} Jul 08 18:33:59.032791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.032678 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf" event={"ID":"2d46c367-b3ba-419e-9867-91648c413314","Type":"ContainerDied","Data":"34d0867e4cbd599f16767d681b864c1f398c417eaeb01a294566094d953a2dc3"} Jul 08 18:33:59.032791 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.032694 2562 scope.go:117] "RemoveContainer" containerID="3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779" Jul 08 18:33:59.043565 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.043518 2562 scope.go:117] "RemoveContainer" containerID="3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779" Jul 08 18:33:59.044006 ip-10-0-131-126 kubenswrapper[2562]: E0708 18:33:59.043985 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779\": container with ID starting with 3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779 not found: ID does not exist" containerID="3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779" Jul 08 18:33:59.044076 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.044015 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779"} err="failed to get container status \"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779\": rpc error: code = NotFound desc = could not find container \"3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779\": container with ID starting with 3d88d925a8a6b493060debf3504965cf8c9885db4dfec5a86219c821eaa16779 not found: ID does not exist" Jul 08 18:33:59.053650 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.053617 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:33:59.055405 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:33:59.055385 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3ef261af/e2e-lifecycle-model-3ef261af-kserve-5cc59d864c-9xmvf"] Jul 08 18:34:00.190040 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:34:00.190004 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d46c367-b3ba-419e-9867-91648c413314" path="/var/lib/kubelet/pods/2d46c367-b3ba-419e-9867-91648c413314/volumes" Jul 08 18:34:23.166965 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:34:23.166929 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:34:36.778275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:34:36.778190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:34:48.877042 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:34:48.876998 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:35:38.969283 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:38.969242 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:35:50.096756 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:50.096721 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:35:59.449589 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.449552 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:35:59.450128 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.450109 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d46c367-b3ba-419e-9867-91648c413314" containerName="main" Jul 08 18:35:59.450173 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.450133 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d46c367-b3ba-419e-9867-91648c413314" containerName="main" Jul 08 18:35:59.450260 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.450248 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d46c367-b3ba-419e-9867-91648c413314" containerName="main" Jul 08 18:35:59.453793 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.453774 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.457149 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.457129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-94205f\"/\"default-dockercfg-wjhsc\"" Jul 08 18:35:59.457281 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.457157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-94205f\"/\"openshift-service-ca.crt\"" Jul 08 18:35:59.457281 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.457130 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-94205f\"/\"test-model-94205f-kserve-self-signed-certs\"" Jul 08 18:35:59.457281 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.457217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-94205f\"/\"kube-root-ca.crt\"" Jul 08 18:35:59.461683 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.461662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:35:59.581954 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.581919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brts8\" (UniqueName: \"kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.582111 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.581962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.582111 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.582087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.582202 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.582126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.582202 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.582161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.582202 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.582188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683124 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683297 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683297 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683151 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brts8\" (UniqueName: \"kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683297 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683498 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683498 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683614 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683587 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683754 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683735 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.683796 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.683744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.685655 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.685627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.685827 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.685811 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.691939 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.691917 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brts8\" (UniqueName: \"kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8\") pod \"test-model-94205f-kserve-78b48bbbcc-cv2rf\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.763558 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.763497 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:35:59.895541 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:35:59.895494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:35:59.897787 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:35:59.897762 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67382807_70dd_4be6_848c_7e4e34583bda.slice/crio-1c4c5da215cb4f71790ac60a04047b5614e9a483990d9e0475cea3dc208d261c WatchSource:0}: Error finding container 1c4c5da215cb4f71790ac60a04047b5614e9a483990d9e0475cea3dc208d261c: Status 404 returned error can't find the container with id 1c4c5da215cb4f71790ac60a04047b5614e9a483990d9e0475cea3dc208d261c Jul 08 18:36:00.574604 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:00.574571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" event={"ID":"67382807-70dd-4be6-848c-7e4e34583bda","Type":"ContainerStarted","Data":"6ac41abbf6c31c02d05af606fb103196c191f9e470c63802e955625f65ea1c46"} Jul 08 18:36:00.574604 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:00.574610 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" event={"ID":"67382807-70dd-4be6-848c-7e4e34583bda","Type":"ContainerStarted","Data":"1c4c5da215cb4f71790ac60a04047b5614e9a483990d9e0475cea3dc208d261c"} Jul 08 18:36:00.591433 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:00.591391 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" podStartSLOduration=1.591377476 podStartE2EDuration="1.591377476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:36:00.590592403 +0000 UTC m=+1874.930036414" watchObservedRunningTime="2026-07-08 18:36:00.591377476 +0000 UTC m=+1874.930821514" Jul 08 18:36:03.489258 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:03.489221 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:36:09.764023 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:09.763936 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:36:09.764023 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:09.763987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:36:09.776287 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:09.776264 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:36:10.632351 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:10.632322 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:36:22.832801 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.832766 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:36:22.836078 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.836056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.838330 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.838307 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-44b28a\"/\"kube-root-ca.crt\"" Jul 08 18:36:22.839203 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.839183 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-44b28a\"/\"test-model-44b28a-kserve-self-signed-certs\"" Jul 08 18:36:22.839537 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.839520 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-44b28a\"/\"default-dockercfg-cm7w6\"" Jul 08 18:36:22.839625 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.839547 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-44b28a\"/\"openshift-service-ca.crt\"" Jul 08 18:36:22.849807 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.848390 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:36:22.985200 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.985200 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.985371 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.985371 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.985371 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75479\" (UniqueName: \"kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:22.985371 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:22.985342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086219 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086219 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086219 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086219 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086539 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75479\" (UniqueName: \"kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086539 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086663 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086714 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086649 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.086751 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.086725 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.088584 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.088558 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.088954 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.088935 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.095705 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.095683 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75479\" (UniqueName: \"kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479\") pod \"test-model-44b28a-kserve-b9df5bccf-7m5vn\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.145743 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.145721 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:23.275738 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.275711 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:36:23.277414 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:36:23.277390 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1bafeda_e873_4179_a4c1_e36fed15d1a4.slice/crio-95e56a0cd0d77199fa724c8409aa5b6c8850ea8e6a51ecac6744e62c8c5b72df WatchSource:0}: Error finding container 95e56a0cd0d77199fa724c8409aa5b6c8850ea8e6a51ecac6744e62c8c5b72df: Status 404 returned error can't find the container with id 95e56a0cd0d77199fa724c8409aa5b6c8850ea8e6a51ecac6744e62c8c5b72df Jul 08 18:36:23.682818 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.682783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" event={"ID":"e1bafeda-e873-4179-a4c1-e36fed15d1a4","Type":"ContainerStarted","Data":"7c4b4aaa35885177b84ac523656b0e2e1a06bad51516324ca2b671f2599f2788"} Jul 08 18:36:23.682981 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.682829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" event={"ID":"e1bafeda-e873-4179-a4c1-e36fed15d1a4","Type":"ContainerStarted","Data":"95e56a0cd0d77199fa724c8409aa5b6c8850ea8e6a51ecac6744e62c8c5b72df"} Jul 08 18:36:23.699468 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:23.699406 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" podStartSLOduration=1.699392555 podStartE2EDuration="1.699392555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:36:23.698668414 +0000 UTC m=+1898.038112422" watchObservedRunningTime="2026-07-08 18:36:23.699392555 +0000 UTC m=+1898.038836564" Jul 08 18:36:26.575997 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:26.575968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:36:33.146720 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:33.146681 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:33.146720 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:33.146723 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:33.159110 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:33.159080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:33.739719 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:33.739692 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:36:39.976703 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:39.976666 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:36:57.712100 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:57.712068 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:36:58.662654 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:36:58.662624 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:37:02.479946 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:02.479909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:37:12.734129 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:12.734089 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" podUID="67382807-70dd-4be6-848c-7e4e34583bda" containerName="main" containerID="cri-o://6ac41abbf6c31c02d05af606fb103196c191f9e470c63802e955625f65ea1c46" gracePeriod=45 Jul 08 18:37:12.912316 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:12.912287 2562 generic.go:356] "Generic (PLEG): container finished" podID="67382807-70dd-4be6-848c-7e4e34583bda" containerID="6ac41abbf6c31c02d05af606fb103196c191f9e470c63802e955625f65ea1c46" exitCode=0 Jul 08 18:37:12.912472 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:12.912347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" event={"ID":"67382807-70dd-4be6-848c-7e4e34583bda","Type":"ContainerDied","Data":"6ac41abbf6c31c02d05af606fb103196c191f9e470c63802e955625f65ea1c46"} Jul 08 18:37:12.986438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:12.986371 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:37:13.133017 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.132975 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133017 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133028 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133072 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133098 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133131 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brts8\" (UniqueName: \"kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133280 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133180 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs\") pod \"67382807-70dd-4be6-848c-7e4e34583bda\" (UID: \"67382807-70dd-4be6-848c-7e4e34583bda\") " Jul 08 18:37:13.133508 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133373 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:13.133508 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133389 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache" (OuterVolumeSpecName: "model-cache") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:13.133508 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133402 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home" (OuterVolumeSpecName: "home") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:13.133634 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133568 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-tmp-dir\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.133634 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133587 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-model-cache\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.133634 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.133600 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-home\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.135547 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.135518 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:37:13.135547 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.135537 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm" (OuterVolumeSpecName: "dshm") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:13.135719 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.135570 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8" (OuterVolumeSpecName: "kube-api-access-brts8") pod "67382807-70dd-4be6-848c-7e4e34583bda" (UID: "67382807-70dd-4be6-848c-7e4e34583bda"). InnerVolumeSpecName "kube-api-access-brts8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:13.234860 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.234838 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brts8\" (UniqueName: \"kubernetes.io/projected/67382807-70dd-4be6-848c-7e4e34583bda-kube-api-access-brts8\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.234860 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.234858 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67382807-70dd-4be6-848c-7e4e34583bda-tls-certs\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.234997 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.234868 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67382807-70dd-4be6-848c-7e4e34583bda-dshm\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:13.683835 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.683781 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerName="main" containerID="cri-o://7c4b4aaa35885177b84ac523656b0e2e1a06bad51516324ca2b671f2599f2788" gracePeriod=45 Jul 08 18:37:13.729521 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.729448 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/ready\": dial tcp 10.134.0.32:8000: connect: connection refused" Jul 08 18:37:13.918901 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.918759 2562 generic.go:356] "Generic (PLEG): container finished" podID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerID="7c4b4aaa35885177b84ac523656b0e2e1a06bad51516324ca2b671f2599f2788" exitCode=0 Jul 08 18:37:13.918901 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.918845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" event={"ID":"e1bafeda-e873-4179-a4c1-e36fed15d1a4","Type":"ContainerDied","Data":"7c4b4aaa35885177b84ac523656b0e2e1a06bad51516324ca2b671f2599f2788"} Jul 08 18:37:13.920431 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.920412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" event={"ID":"67382807-70dd-4be6-848c-7e4e34583bda","Type":"ContainerDied","Data":"1c4c5da215cb4f71790ac60a04047b5614e9a483990d9e0475cea3dc208d261c"} Jul 08 18:37:13.920647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.920473 2562 scope.go:117] "RemoveContainer" containerID="6ac41abbf6c31c02d05af606fb103196c191f9e470c63802e955625f65ea1c46" Jul 08 18:37:13.920647 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.920484 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf" Jul 08 18:37:13.937607 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.937591 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:37:13.947045 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.947024 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:37:13.952603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:13.952586 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-94205f/test-model-94205f-kserve-78b48bbbcc-cv2rf"] Jul 08 18:37:14.045171 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045142 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045310 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045182 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045310 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045226 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045310 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045244 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045310 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75479\" (UniqueName: \"kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045552 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045332 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm\") pod \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\" (UID: \"e1bafeda-e873-4179-a4c1-e36fed15d1a4\") " Jul 08 18:37:14.045552 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045422 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:14.045651 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045564 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home" (OuterVolumeSpecName: "home") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:14.045651 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045577 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache" (OuterVolumeSpecName: "model-cache") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:14.045743 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045680 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-home\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.045743 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045698 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-model-cache\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.045743 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.045713 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tmp-dir\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.047644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.047623 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm" (OuterVolumeSpecName: "dshm") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:37:14.047644 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.047629 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:37:14.047774 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.047696 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479" (OuterVolumeSpecName: "kube-api-access-75479") pod "e1bafeda-e873-4179-a4c1-e36fed15d1a4" (UID: "e1bafeda-e873-4179-a4c1-e36fed15d1a4"). InnerVolumeSpecName "kube-api-access-75479". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:37:14.146809 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.146782 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75479\" (UniqueName: \"kubernetes.io/projected/e1bafeda-e873-4179-a4c1-e36fed15d1a4-kube-api-access-75479\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.146809 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.146807 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1bafeda-e873-4179-a4c1-e36fed15d1a4-dshm\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.146949 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.146817 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1bafeda-e873-4179-a4c1-e36fed15d1a4-tls-certs\") on node \"ip-10-0-131-126.ec2.internal\" DevicePath \"\"" Jul 08 18:37:14.189660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.189605 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67382807-70dd-4be6-848c-7e4e34583bda" path="/var/lib/kubelet/pods/67382807-70dd-4be6-848c-7e4e34583bda/volumes" Jul 08 18:37:14.927861 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.927826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" event={"ID":"e1bafeda-e873-4179-a4c1-e36fed15d1a4","Type":"ContainerDied","Data":"95e56a0cd0d77199fa724c8409aa5b6c8850ea8e6a51ecac6744e62c8c5b72df"} Jul 08 18:37:14.927861 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.927857 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn" Jul 08 18:37:14.928326 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.927881 2562 scope.go:117] "RemoveContainer" containerID="7c4b4aaa35885177b84ac523656b0e2e1a06bad51516324ca2b671f2599f2788" Jul 08 18:37:14.944505 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.944481 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:37:14.949120 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:14.949095 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-44b28a/test-model-44b28a-kserve-b9df5bccf-7m5vn"] Jul 08 18:37:16.188257 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:16.188225 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" path="/var/lib/kubelet/pods/e1bafeda-e873-4179-a4c1-e36fed15d1a4/volumes" Jul 08 18:37:45.680511 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:37:45.680429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:38:00.168686 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:38:00.168652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:38:12.094625 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:38:12.094592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:38:23.371750 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:38:23.371720 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:38:36.390367 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:38:36.390330 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:38:47.789519 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:38:47.789481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:41:53.677742 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:41:53.677707 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:42:02.389773 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:02.389739 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-vrtf8"] Jul 08 18:42:09.595411 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:09.595322 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-286x9_810e8946-94c1-46d4-b6f6-3f2500d7a501/manager/0.log" Jul 08 18:42:10.049735 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:10.049706 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-d6hzx_b14fb214-15ee-4e81-bf01-19f15167940a/copy-manifests/0.log" Jul 08 18:42:10.057121 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:10.057091 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-d6hzx_b14fb214-15ee-4e81-bf01-19f15167940a/manager/0.log" Jul 08 18:42:11.962613 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:11.962584 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pmn7s_cb31782c-5454-4762-8459-6afd4600785c/kuadrant-console-plugin/0.log" Jul 08 18:42:12.337816 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:12.337748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vrtf8_db112ac4-1357-49e9-a228-df2a620701ee/limitador/0.log" Jul 08 18:42:13.195039 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:13.195012 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-9hjnd_236bd738-f67c-4fac-8bff-c14ae5ad22dc/kube-auth-proxy/0.log" Jul 08 18:42:14.159360 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:14.159319 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bvsk2_522f575e-d081-41e1-aec9-f961540cd199/main/0.log" Jul 08 18:42:14.522080 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:14.522045 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4rt2d_36d57ea2-4417-46fd-9431-6f2a221f4a55/main/0.log" Jul 08 18:42:14.637797 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:14.637770 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-rmv6k_f3d708e4-d4b4-429f-bcbc-9d57b3ea47ac/main/0.log" Jul 08 18:42:21.490231 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:21.490194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2tv7h_d6085b44-3dca-430b-a48d-0b8c9dcc8942/global-pull-secret-syncer/0.log" Jul 08 18:42:21.660502 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:21.660472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d5slx_0c87a352-a206-4468-bda7-9cecf2f87adb/konnectivity-agent/0.log" Jul 08 18:42:21.729498 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:21.729471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-126.ec2.internal_a9496e6239949feac3069ae863246fd9/haproxy/0.log" Jul 08 18:42:26.557752 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:26.557719 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-pmn7s_cb31782c-5454-4762-8459-6afd4600785c/kuadrant-console-plugin/0.log" Jul 08 18:42:26.826985 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:26.826912 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-vrtf8_db112ac4-1357-49e9-a228-df2a620701ee/limitador/0.log" Jul 08 18:42:28.422626 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.422602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/alertmanager/0.log" Jul 08 18:42:28.443583 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.443559 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/config-reloader/0.log" Jul 08 18:42:28.464365 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.464347 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/kube-rbac-proxy-web/0.log" Jul 08 18:42:28.485113 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.485091 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/kube-rbac-proxy/0.log" Jul 08 18:42:28.510138 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.510116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/kube-rbac-proxy-metric/0.log" Jul 08 18:42:28.531002 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.530980 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/prom-label-proxy/0.log" Jul 08 18:42:28.556345 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.556331 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_63d67666-7d25-4f64-ac32-da1b600c573c/init-config-reloader/0.log" Jul 08 18:42:28.759377 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.759356 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-tmbxv_40694806-ab82-4e2e-b797-b8a8a8c9730b/monitoring-plugin/0.log" Jul 08 18:42:28.983057 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:28.983034 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8285_b5990daa-0737-479d-8df3-638a380392dc/node-exporter/0.log" Jul 08 18:42:29.007019 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.006996 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8285_b5990daa-0737-479d-8df3-638a380392dc/kube-rbac-proxy/0.log" Jul 08 18:42:29.029952 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.029927 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8285_b5990daa-0737-479d-8df3-638a380392dc/init-textfile/0.log" Jul 08 18:42:29.269785 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.269762 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/prometheus/0.log" Jul 08 18:42:29.310385 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.310356 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/config-reloader/0.log" Jul 08 18:42:29.337132 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.337111 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/thanos-sidecar/0.log" Jul 08 18:42:29.360603 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.360577 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/kube-rbac-proxy-web/0.log" Jul 08 18:42:29.390446 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.390426 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/kube-rbac-proxy/0.log" Jul 08 18:42:29.417382 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.417359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/kube-rbac-proxy-thanos/0.log" Jul 08 18:42:29.444028 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.444007 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88dda251-d03d-4683-aa67-75eca4aeb926/init-config-reloader/0.log" Jul 08 18:42:29.475660 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.475635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-z529q_3a4dfb11-bdce-43ec-a938-730bac5de22d/prometheus-operator/0.log" Jul 08 18:42:29.498275 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.498251 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-z529q_3a4dfb11-bdce-43ec-a938-730bac5de22d/kube-rbac-proxy/0.log" Jul 08 18:42:29.564844 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.564757 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d7b9f98b7-lnq6w_9ef0a0b4-e098-44d2-b33e-dca6bc120f0a/telemeter-client/0.log" Jul 08 18:42:29.588476 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.588434 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d7b9f98b7-lnq6w_9ef0a0b4-e098-44d2-b33e-dca6bc120f0a/reload/0.log" Jul 08 18:42:29.614502 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.614482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d7b9f98b7-lnq6w_9ef0a0b4-e098-44d2-b33e-dca6bc120f0a/kube-rbac-proxy/0.log" Jul 08 18:42:29.649764 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.649738 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/thanos-query/0.log" Jul 08 18:42:29.674007 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.673990 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/kube-rbac-proxy-web/0.log" Jul 08 18:42:29.701567 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.701551 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/kube-rbac-proxy/0.log" Jul 08 18:42:29.726541 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.726524 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/prom-label-proxy/0.log" Jul 08 18:42:29.750188 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.750173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/kube-rbac-proxy-rules/0.log" Jul 08 18:42:29.774897 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:29.774878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-78fdbb6986-6psdj_f0ad3018-9d1e-4646-8884-93ef62050456/kube-rbac-proxy-metrics/0.log" Jul 08 18:42:31.626047 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626012 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9"] Jul 08 18:42:31.626438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626405 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67382807-70dd-4be6-848c-7e4e34583bda" containerName="main" Jul 08 18:42:31.626438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626419 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67382807-70dd-4be6-848c-7e4e34583bda" containerName="main" Jul 08 18:42:31.626438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626428 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerName="main" Jul 08 18:42:31.626438 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626433 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerName="main" Jul 08 18:42:31.626593 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626522 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1bafeda-e873-4179-a4c1-e36fed15d1a4" containerName="main" Jul 08 18:42:31.626593 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.626535 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67382807-70dd-4be6-848c-7e4e34583bda" containerName="main" Jul 08 18:42:31.629781 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.629762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.632145 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.632125 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9mvv9\"/\"kube-root-ca.crt\"" Jul 08 18:42:31.633079 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.633061 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9mvv9\"/\"openshift-service-ca.crt\"" Jul 08 18:42:31.633192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.633076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9mvv9\"/\"default-dockercfg-tjq4t\"" Jul 08 18:42:31.642514 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.642494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9"] Jul 08 18:42:31.707421 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.707392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-sys\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.707570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.707426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-lib-modules\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.707570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.707486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrvk\" (UniqueName: \"kubernetes.io/projected/93641387-5e05-43a5-b1a1-b86091ca7546-kube-api-access-vwrvk\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.707570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.707510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-podres\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.707570 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.707526 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-proc\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808344 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-podres\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808344 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-proc\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-sys\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-lib-modules\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrvk\" (UniqueName: \"kubernetes.io/projected/93641387-5e05-43a5-b1a1-b86091ca7546-kube-api-access-vwrvk\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-podres\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808540 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-proc\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808773 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808541 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-sys\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.808773 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.808610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93641387-5e05-43a5-b1a1-b86091ca7546-lib-modules\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.817650 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.817627 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrvk\" (UniqueName: \"kubernetes.io/projected/93641387-5e05-43a5-b1a1-b86091ca7546-kube-api-access-vwrvk\") pod \"perf-node-gather-daemonset-dckr9\" (UID: \"93641387-5e05-43a5-b1a1-b86091ca7546\") " pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:31.940042 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:31.940018 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:32.067228 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.067205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9"] Jul 08 18:42:32.070110 ip-10-0-131-126 kubenswrapper[2562]: W0708 18:42:32.070087 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod93641387_5e05_43a5_b1a1_b86091ca7546.slice/crio-dad92192f49220919bb209cbe6fc139919f276ea4ab81e65fb3e4bdcaed4c571 WatchSource:0}: Error finding container dad92192f49220919bb209cbe6fc139919f276ea4ab81e65fb3e4bdcaed4c571: Status 404 returned error can't find the container with id dad92192f49220919bb209cbe6fc139919f276ea4ab81e65fb3e4bdcaed4c571 Jul 08 18:42:32.248140 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.248061 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-zvtlz_e4bcf3ba-96ea-4462-959b-dbbee92715d5/download-server/0.log" Jul 08 18:42:32.355192 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.355163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" event={"ID":"93641387-5e05-43a5-b1a1-b86091ca7546","Type":"ContainerStarted","Data":"6fb6b76f8b448c0415351b36b6292af32df542dc8f8ebb20a934523a611b0beb"} Jul 08 18:42:32.355364 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.355208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" event={"ID":"93641387-5e05-43a5-b1a1-b86091ca7546","Type":"ContainerStarted","Data":"dad92192f49220919bb209cbe6fc139919f276ea4ab81e65fb3e4bdcaed4c571"} Jul 08 18:42:32.355364 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.355303 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:32.371997 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:32.371957 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" podStartSLOduration=1.371943595 podStartE2EDuration="1.371943595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:42:32.37082767 +0000 UTC m=+2266.710271677" watchObservedRunningTime="2026-07-08 18:42:32.371943595 +0000 UTC m=+2266.711387603" Jul 08 18:42:33.939634 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:33.939608 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxlzr_6039c0a5-514c-4508-8961-80a26b8ed4e6/dns/0.log" Jul 08 18:42:33.960129 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:33.960110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wxlzr_6039c0a5-514c-4508-8961-80a26b8ed4e6/kube-rbac-proxy/0.log" Jul 08 18:42:34.035953 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:34.035932 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wgwr6_472a4a0a-88f5-4a4b-b095-8cb740e75cfa/dns-node-resolver/0.log" Jul 08 18:42:34.607328 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:34.607298 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-44g8z_19d67e3b-9d7e-4089-ac36-2559bf0caf72/node-ca/0.log" Jul 08 18:42:35.846775 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:35.846746 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-9hjnd_236bd738-f67c-4fac-8bff-c14ae5ad22dc/kube-auth-proxy/0.log" Jul 08 18:42:36.744159 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:36.744128 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vlvs2_2c54a90d-3ccc-4291-a248-2a72dcfc71b2/serve-healthcheck-canary/0.log" Jul 08 18:42:37.394400 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:37.394371 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgwh2_32133831-f978-43c4-94fe-33bfe0490974/kube-rbac-proxy/0.log" Jul 08 18:42:37.416407 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:37.416387 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgwh2_32133831-f978-43c4-94fe-33bfe0490974/exporter/0.log" Jul 08 18:42:37.436851 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:37.436830 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xgwh2_32133831-f978-43c4-94fe-33bfe0490974/extractor/0.log" Jul 08 18:42:38.369557 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:38.369523 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9mvv9/perf-node-gather-daemonset-dckr9" Jul 08 18:42:39.641605 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:39.641568 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-286x9_810e8946-94c1-46d4-b6f6-3f2500d7a501/manager/0.log" Jul 08 18:42:39.848422 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:39.848394 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-d6hzx_b14fb214-15ee-4e81-bf01-19f15167940a/manager/0.log" Jul 08 18:42:39.871689 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:39.871667 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859965ff4-d6hzx_b14fb214-15ee-4e81-bf01-19f15167940a/copy-manifests/0.log" Jul 08 18:42:41.430174 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:41.430108 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68bbd4cf59-dzgbg_3d151273-acd8-40fa-b1f3-2c016254ba4a/manager/0.log" Jul 08 18:42:42.089497 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:42.089465 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-9hjnd_236bd738-f67c-4fac-8bff-c14ae5ad22dc/kube-auth-proxy/0.log" Jul 08 18:42:43.581299 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:43.581271 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6784757fb-9hjnd_236bd738-f67c-4fac-8bff-c14ae5ad22dc/kube-auth-proxy/0.log" Jul 08 18:42:47.945146 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:47.945116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5dhq5_96b06878-7ebd-47c8-9232-2afab2afc896/kube-multus/0.log" Jul 08 18:42:48.386872 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.386798 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/kube-multus-additional-cni-plugins/0.log" Jul 08 18:42:48.411990 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.411967 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/egress-router-binary-copy/0.log" Jul 08 18:42:48.434181 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.434152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/cni-plugins/0.log" Jul 08 18:42:48.457909 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.457887 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/bond-cni-plugin/0.log" Jul 08 18:42:48.481104 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.481084 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/routeoverride-cni/0.log" Jul 08 18:42:48.503854 ip-10-0-131-126 kubenswrapper[2562]: I0708 18:42:48.503828 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kplpd_bd748a79-b017-4b08-8450-5d0734cefe9e/whereabouts-cni-bincopy/0.log"