Jul 08 18:56:53.097155 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:56:53.097166 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:56:53.097173 ip-10-0-142-223 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:56:53.098088 ip-10-0-142-223 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:57:03.303546 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 18:57:03.303560 ip-10-0-142-223 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f64d0d189b8b49dcad5dacbdfdb1df33 -- Jul 08 18:59:01.658609 ip-10-0-142-223 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:59:02.126812 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.122985 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128442 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128455 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128460 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128465 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128469 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128474 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128478 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128482 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128486 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128495 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128499 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128503 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128507 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128511 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128515 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128519 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128524 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128528 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128531 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:02.130546 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128536 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128540 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128543 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128547 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128551 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128555 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128559 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128562 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128566 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128570 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128574 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128578 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128581 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128585 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128589 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128593 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128597 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128603 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128608 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:02.137435 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128612 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128616 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128619 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128624 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128628 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128632 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128635 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128640 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128643 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128647 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128651 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128657 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128662 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128666 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128670 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128675 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128680 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128684 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128688 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:02.138060 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128693 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128697 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128701 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128705 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128709 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128714 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128718 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128723 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128727 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128731 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128735 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128739 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128743 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128746 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128750 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128754 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128758 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128761 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128766 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128769 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:02.138694 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128773 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128777 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128780 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128785 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128788 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128792 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128796 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128801 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128804 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128808 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128812 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128816 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128819 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128823 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128827 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128831 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128835 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128838 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128844 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128849 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:02.139323 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128853 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.128857 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131584 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131601 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131604 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131607 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131611 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131614 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131617 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131620 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131625 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131628 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131631 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131634 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131637 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131639 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131642 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131645 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131647 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:02.139943 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131650 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131653 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131655 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131661 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131663 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131669 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131672 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131675 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131678 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131681 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131684 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131687 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131690 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131692 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131695 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131699 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131706 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131709 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131713 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131716 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:02.141008 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131719 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131722 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131725 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131727 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131730 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131734 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131738 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131743 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131746 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131749 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131751 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131754 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131761 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131764 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131766 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131769 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131772 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131774 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131778 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131781 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:02.141657 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131786 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131789 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131792 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131794 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131797 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131800 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131804 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131809 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131812 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131815 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131818 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131821 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131826 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131829 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131832 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131835 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131838 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131841 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131844 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:02.227239 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131847 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:02.197397 ip-10-0-142-223 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131849 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131853 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131855 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131862 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131865 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131870 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131872 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131876 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131879 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131882 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131884 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131888 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131890 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131893 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131896 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131900 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131906 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131909 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:02.228877 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131912 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131914 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131922 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.131925 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132572 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132578 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132581 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132584 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132587 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132590 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132593 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132596 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132601 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132604 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132606 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132609 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132612 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132617 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132619 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132622 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:02.229896 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132625 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132628 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132630 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132633 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132635 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132641 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132644 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132647 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132650 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132652 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132655 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132658 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132660 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132663 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132666 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132668 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132671 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132676 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132679 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132682 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:02.230496 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132684 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132687 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132689 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132692 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132695 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132698 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132700 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132703 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132705 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132710 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132716 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132719 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132722 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132725 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132728 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132730 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132733 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132736 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132740 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132744 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:02.231212 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132747 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132753 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132756 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132760 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132762 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132765 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132768 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132771 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132774 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132776 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132779 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132782 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132784 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132787 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132792 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132794 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132797 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132800 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132804 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132806 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:02.231918 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132809 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132814 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132816 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132819 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132822 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132824 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132831 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132834 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132837 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132840 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132843 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132846 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132849 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132851 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132854 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132857 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132859 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132862 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132867 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:02.232848 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132870 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132872 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132875 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.132879 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132963 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132973 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132979 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132983 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132988 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132991 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.132997 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133002 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133007 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133010 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133015 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133019 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133022 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133025 2580 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133028 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133032 2580 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133035 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133040 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133044 2580 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:59:02.233782 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133047 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133050 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133054 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133057 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133061 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133064 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133067 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133073 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133076 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133079 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133082 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133085 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133088 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133092 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133095 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133101 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133104 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133107 2580 flags.go:64] FLAG: --enable-server="true" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133110 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133118 2580 flags.go:64] FLAG: --event-burst="100" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133121 2580 flags.go:64] FLAG: --event-qps="50" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133124 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133127 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133131 2580 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:59:02.234822 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133138 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133141 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133144 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133147 2580 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133152 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133156 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133159 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133162 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133167 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133170 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133173 2580 flags.go:64] FLAG: --feature-gates="" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133177 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133180 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133184 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133189 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133199 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133202 2580 flags.go:64] FLAG: --help="false" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133205 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-142-223.ec2.internal" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133209 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133212 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133215 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133219 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133223 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133226 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:59:02.235617 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133231 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133234 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133237 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133242 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133245 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133248 2580 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133251 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133255 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133258 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133263 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133266 2580 flags.go:64] FLAG: --lock-file="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133269 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133272 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133275 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133280 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133283 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133286 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133291 2580 flags.go:64] FLAG: --logging-format="text" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133294 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133298 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133301 2580 flags.go:64] FLAG: --manifest-url="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133304 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133309 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133312 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133316 2580 flags.go:64] FLAG: --max-pods="110" Jul 08 18:59:02.236602 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133321 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133338 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133345 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133348 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133351 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133354 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133357 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133367 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133370 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133373 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133378 2580 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133381 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133385 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133389 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133392 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133396 2580 flags.go:64] FLAG: --port="10250" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133401 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133404 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076cad3f36567dbab" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133408 2580 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133411 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133414 2580 flags.go:64] FLAG: --register-node="true" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133417 2580 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133420 2580 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133424 2580 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:59:02.237663 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133429 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133432 2580 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133435 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133439 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133442 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133445 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133448 2580 flags.go:64] FLAG: --runonce="false" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133451 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133454 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133460 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133463 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133466 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133469 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133472 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133474 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133477 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133480 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133485 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133489 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133493 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133496 2580 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133504 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133509 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133514 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133520 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:59:02.238401 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133524 2580 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133526 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133529 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133532 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133535 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133538 2580 flags.go:64] FLAG: --v="2" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133542 2580 flags.go:64] FLAG: --version="false" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133548 2580 flags.go:64] FLAG: --vmodule="" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133551 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.133555 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133761 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133765 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133770 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133774 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133777 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133780 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133783 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133786 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133789 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133792 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133797 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133800 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:02.239429 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133803 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133805 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133808 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133812 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133815 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133818 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133824 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133826 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133831 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133834 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133837 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133842 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133845 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133848 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133850 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133853 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133856 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133858 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133861 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133864 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:02.240064 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133867 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133869 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133872 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133875 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133880 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133883 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133885 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133888 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133891 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133893 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133896 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133898 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133901 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133903 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133906 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133910 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133913 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133918 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133920 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133923 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:04.180932 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133927 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133930 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133933 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133936 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133939 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133941 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133944 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133947 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133949 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133954 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133957 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133959 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133962 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133965 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133969 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133973 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133976 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133979 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133982 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:04.184111 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133984 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133987 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133992 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133995 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.133998 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134001 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134003 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134006 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134010 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134013 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134016 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134019 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134021 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134025 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134031 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134034 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134037 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134039 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134042 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:04.184721 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134045 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134047 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134050 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134053 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134055 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134058 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134062 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134065 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.134071 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.135048 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.144519 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.144534 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.144552 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.144564 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:59:04.185360 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144620 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144625 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144629 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144632 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144635 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144638 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144640 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144643 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144646 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144649 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144651 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144654 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144657 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144659 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144663 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144667 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144670 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144673 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144677 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:04.185754 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144681 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144683 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144687 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144689 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144692 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144695 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144697 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144700 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144703 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144706 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144709 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144712 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144715 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144717 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144720 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144723 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144726 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144728 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144732 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144735 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:04.186322 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144737 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144740 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144743 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144745 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144748 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144751 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144753 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144756 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144758 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144761 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144764 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144767 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144770 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144773 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144775 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144778 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144780 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144783 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144786 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144789 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:04.186857 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144791 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144794 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144797 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144799 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144802 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144804 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144806 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144809 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144811 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144814 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144817 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144821 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144824 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144828 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144831 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144834 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144836 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144840 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144842 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:04.187450 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144845 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144848 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144851 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144854 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144857 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144860 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144862 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144866 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144868 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144871 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144873 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144876 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144879 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144882 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144884 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144887 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144889 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144892 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144895 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144897 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:04.187950 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.144900 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.144905 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145010 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145015 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145018 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145021 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145023 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145026 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145028 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145031 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145034 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145037 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145040 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145042 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145045 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:59:04.188595 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145047 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145050 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145052 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145055 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145057 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145060 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145062 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145065 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145067 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145070 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145073 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145076 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145078 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145081 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145083 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145086 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145088 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145091 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145093 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145096 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:59:04.189016 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145098 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145101 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145105 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145109 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145112 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145114 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145117 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145119 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145122 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145124 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145127 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145130 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145132 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145135 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145137 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145140 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145144 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145147 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145150 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:59:04.189592 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145153 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145155 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145159 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145162 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145166 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145169 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145172 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145175 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145177 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145179 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145182 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145185 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145187 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145190 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145193 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145195 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145198 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145201 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145203 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145206 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:59:04.190093 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145209 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145212 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145214 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145217 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145219 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145224 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145226 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145229 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145234 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145236 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145239 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145242 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145244 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145247 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145250 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145252 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145255 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145258 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145261 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145264 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:59:04.190708 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145266 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145269 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145271 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145274 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145276 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145279 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:02.145281 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.145286 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.145408 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.147443 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.148931 2580 server.go:1019] "Starting client certificate rotation" Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.149033 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.149070 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:59:04.191241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.152948 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.154173 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.161833 2580 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.163749 2580 log.go:25] "Validated CRI v1 image API" Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.164573 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.169965 2580 fs.go:135] Filesystem UUIDs: map[0c88e719-3b68-46f6-be44-e48f9b6f5c3c:/dev/nvme0n1p3 3d389d44-a100-4360-a5a8-88f336a834ec:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 08 18:59:04.191689 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.169979 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.176012 2580 manager.go:216] Machine: {Timestamp:2026-07-08 18:59:02.174769812 +0000 UTC m=+0.389630144 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108616 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec276f5b366d2b58343e866916efe4ec SystemUUID:ec276f5b-366d-2b58-343e-866916efe4ec BootID:f64d0d18-9b8b-49dc-ad5d-acbdfdb1df33 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3e:ee:fd:97:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3e:ee:fd:97:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:86:85:fd:d8:c1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.176139 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.176209 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.177982 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.178003 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.178646 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.178655 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.178663 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.179199 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.180902 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183346 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183741 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183852 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183878 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183887 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183903 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:59:04.191953 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183912 2580 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.183921 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.184199 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.184213 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.189109 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.190863 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.190890 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192123 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192137 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192143 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192149 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192164 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192174 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192186 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192191 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192198 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192204 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192212 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192218 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192848 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.192854 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.195907 2580 server.go:1268] "Started kubelet" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.196528 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.196462 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.196611 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.196629 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.196637 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.196633 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.198250 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.199130 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.201658 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.203887 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.203892 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.203945 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204518 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204521 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204547 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204661 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204669 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204816 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204833 2580 factory.go:55] Registering systemd factory Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.204843 2580 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.205021 2580 factory.go:153] Registering CRI-O factory Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.205029 2580 factory.go:223] Registration of the crio container factory successfully Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.205045 2580 factory.go:103] Registering Raw factory Jul 08 18:59:04.192484 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.205053 2580 manager.go:1196] Started watching for new ooms in manager Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.203313 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-223.ec2.internal.18c06594cc4cfcda default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-223.ec2.internal,UID:ip-10-0-142-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-223.ec2.internal,},FirstTimestamp:2026-07-08 18:59:02.19587913 +0000 UTC m=+0.410739462,LastTimestamp:2026-07-08 18:59:02.19587913 +0000 UTC m=+0.410739462,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-223.ec2.internal,}" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.205469 2580 manager.go:349] Starting recovery of all containers Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.211961 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.218551 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlds6" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.219144 2580 manager.go:354] Recovery completed Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.225050 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlds6" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.227127 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.228063 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.232343 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.234543 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.234571 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.234585 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236194 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236202 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236205 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236285 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236939 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236970 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.236986 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.237684 2580 policy_none.go:49] "None policy: Start" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.237698 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.237711 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.237690 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-223.ec2.internal.18c06594ce9b25d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-223.ec2.internal,UID:ip-10-0-142-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-223.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-223.ec2.internal,},FirstTimestamp:2026-07-08 18:59:02.234555862 +0000 UTC m=+0.449416194,LastTimestamp:2026-07-08 18:59:02.234555862 +0000 UTC m=+0.449416194,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-223.ec2.internal,}" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.238915 2580 policy_none.go:47] "Start" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.271851 2580 manager.go:337] "Starting Device Plugin manager" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.271993 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272005 2580 server.go:87] "Starting device plugin registration server" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272185 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272196 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272269 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272357 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:02.272366 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:59:04.193685 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.272797 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:02.272847 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.159484 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.160569 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.160590 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.160599 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.160625 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.170793 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.170811 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-223.ec2.internal\": node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.194919 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.188516 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.207653 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.207681 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.207697 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.207732 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.209894 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.288827 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.308193 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal"] Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.308264 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.309822 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.309847 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.309868 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.311390 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.311501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.311532 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312146 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312161 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312186 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312199 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312165 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.312251 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.313814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.313834 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.314446 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.314466 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.314476 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.328809 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.ec2.internal\" not found" node="ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.356904 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.334763 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.ec2.internal\" not found" node="ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.389051 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.393373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/872c3c853569ccf8888d4463b966dc40-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.ec2.internal\" (UID: \"872c3c853569ccf8888d4463b966dc40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.393396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.393414 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.489229 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/872c3c853569ccf8888d4463b966dc40-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.ec2.internal\" (UID: \"872c3c853569ccf8888d4463b966dc40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493621 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/872c3c853569ccf8888d4463b966dc40-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.ec2.internal\" (UID: \"872c3c853569ccf8888d4463b966dc40\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493640 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.515220 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.493654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c32a2d252b8e16f47064d5493ac88b8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal\" (UID: \"1c32a2d252b8e16f47064d5493ac88b8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.629793 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.589551 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.632228 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.631945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.843870 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.637921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:04.843870 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.690656 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.843870 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.791073 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.967158 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.891876 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:04.967158 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.935276 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:59:04.967158 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:04.942253 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:59:05.131022 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:04.992886 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:05.131022 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.014562 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 18:54:04 +0000 UTC" deadline="2028-01-01 03:45:11.020447108 +0000 UTC" Jul 08 18:59:05.131022 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.014583 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12992h46m6.005866884s" Jul 08 18:59:05.131022 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.042747 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:59:05.131022 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.093596 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.193792 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:05.197260 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c32a2d252b8e16f47064d5493ac88b8.slice/crio-57b93de66cfe593f6293bd5dad29a328b59ff41833860be12aa6e52fb975e1db WatchSource:0}: Error finding container 57b93de66cfe593f6293bd5dad29a328b59ff41833860be12aa6e52fb975e1db: Status 404 returned error can't find the container with id 57b93de66cfe593f6293bd5dad29a328b59ff41833860be12aa6e52fb975e1db Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:05.197607 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod872c3c853569ccf8888d4463b966dc40.slice/crio-6393631a17f18297d1237c9a56b82144f563f02b0e4ca95ea3c52a592e9b6d7c WatchSource:0}: Error finding container 6393631a17f18297d1237c9a56b82144f563f02b0e4ca95ea3c52a592e9b6d7c: Status 404 returned error can't find the container with id 6393631a17f18297d1237c9a56b82144f563f02b0e4ca95ea3c52a592e9b6d7c Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.202163 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.210857 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" event={"ID":"872c3c853569ccf8888d4463b966dc40","Type":"ContainerStarted","Data":"6393631a17f18297d1237c9a56b82144f563f02b0e4ca95ea3c52a592e9b6d7c"} Jul 08 18:59:05.292038 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.211856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" event={"ID":"1c32a2d252b8e16f47064d5493ac88b8","Type":"ContainerStarted","Data":"57b93de66cfe593f6293bd5dad29a328b59ff41833860be12aa6e52fb975e1db"} Jul 08 18:59:05.296151 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.295974 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:05.431639 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.312137 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:59:05.431639 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.396544 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-223.ec2.internal\" not found" Jul 08 18:59:05.565761 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.484958 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:59:05.565761 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.491312 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" Jul 08 18:59:05.565761 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.505248 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:59:05.565761 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.506319 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" Jul 08 18:59:05.565761 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.514260 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:59:05.941841 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.941770 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:59:05.971874 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.971832 2580 apiserver.go:52] "Watching apiserver" Jul 08 18:59:05.980420 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.980251 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:59:05.980684 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.980659 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-cgfgt","kube-system/global-pull-secret-syncer-hclm2","kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr","openshift-image-registry/node-ca-655q2","openshift-multus/multus-additional-cni-plugins-hdhlq","openshift-multus/multus-qjc6t","kube-system/konnectivity-agent-lvtvj","openshift-cluster-node-tuning-operator/tuned-xz2x8","openshift-dns/node-resolver-rtnkj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal","openshift-multus/network-metrics-daemon-9gmnq","openshift-network-diagnostics/network-check-target-ljgz5","openshift-network-operator/iptables-alerter-prwfw"] Jul 08 18:59:05.982311 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.982252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjc6t" Jul 08 18:59:05.983557 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.983527 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:05.983667 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.983601 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:05.984695 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.984525 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:05.985574 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.985536 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.985680 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.985622 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wvhkg\"" Jul 08 18:59:05.985889 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.985747 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.985889 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.985797 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987057 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987098 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987111 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987257 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s7vb8\"" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987380 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.988369 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.987958 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:59:05.990137 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.990091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:05.990933 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.990780 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.990933 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.990803 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.990933 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.990815 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:59:05.991112 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.991094 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87bt5\"" Jul 08 18:59:05.992339 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.991855 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:59:05.992339 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.991865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:05.992706 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.992686 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.992855 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.992839 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.992939 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.992924 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:59:05.993804 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.993651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8brz8\"" Jul 08 18:59:05.994509 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.993925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:05.994509 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.994332 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:59:05.994509 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.994335 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7w88f\"" Jul 08 18:59:05.994743 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.994727 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:59:05.995494 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.995476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:05.996531 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.996510 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4ql8f\"" Jul 08 18:59:05.996624 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.996580 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.996867 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.996845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:05.996956 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:05.996936 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:05.997514 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.997495 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.997901 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.997878 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:59:05.998425 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.998408 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:59:05.998719 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.998694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:05.999430 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.999407 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:59:05.999509 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.999491 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-997l4\"" Jul 08 18:59:06.000011 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:05.999985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.000563 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.000506 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:59:06.001088 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001062 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:59:06.001344 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.001445 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.001387 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:06.001501 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001455 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:59:06.001565 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-multus\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.001604 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-daemon-config\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.001670 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-etc-kubernetes\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.001670 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:06.001670 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-etc-selinux\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-sys-fs\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001682 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dxckj\"" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-kubelet\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-socket-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-os-release\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.001810 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001791 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-os-release\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-socket-dir-parent\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-host-slash\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.001985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-device-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-conf-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-system-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwc7t\" (UniqueName: \"kubernetes.io/projected/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-kube-api-access-xwc7t\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n2fq\" (UniqueName: \"kubernetes.io/projected/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kube-api-access-8n2fq\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58595100-3eee-4a74-9476-e81e8609049c-serviceca\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002142 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8pq\" (UniqueName: \"kubernetes.io/projected/58595100-3eee-4a74-9476-e81e8609049c-kube-api-access-nl8pq\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-cnibin\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-bin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-iptables-alerter-script\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.002265 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-system-cni-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-k8s-cni-cncf-io\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-hostroot\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002438 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-registration-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58595100-3eee-4a74-9476-e81e8609049c-host\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcbd7\" (UniqueName: \"kubernetes.io/projected/75397fac-717f-42a0-a024-70a29cc0357d-kube-api-access-bcbd7\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cni-binary-copy\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002588 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-netns\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-multus-certs\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvps6\" (UniqueName: \"kubernetes.io/projected/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-kube-api-access-jvps6\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002705 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cnibin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.002996 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.002680 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:59:06.003613 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.003426 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tpkr8\"" Jul 08 18:59:06.007368 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.007348 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:59:06.041371 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.041352 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6hl8r" Jul 08 18:59:06.051999 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.051634 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6hl8r" Jul 08 18:59:06.092488 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.092468 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-netns\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-multus-certs\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvps6\" (UniqueName: \"kubernetes.io/projected/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-kube-api-access-jvps6\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-lib-modules\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-var-lib-kubelet\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-log-socket\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-env-overrides\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cnibin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-etc-kubernetes\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-systemd\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-tuned\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-node-log\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.105685 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-slash\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-var-lib-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270c8cba-3f78-4663-85e1-ba8d92d58158-hosts-file\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-os-release\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-os-release\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-host-slash\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-kubelet-config\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103783 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-etc-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c8cba-3f78-4663-85e1-ba8d92d58158-tmp-dir\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-conf-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-system-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-conf\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103918 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58595100-3eee-4a74-9476-e81e8609049c-serviceca\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-cnibin\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.103985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61016499-c57a-4ff7-8131-62c1ef94de54-agent-certs\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysconfig\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.106614 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-systemd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-system-cni-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-k8s-cni-cncf-io\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-host\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61016499-c57a-4ff7-8131-62c1ef94de54-konnectivity-ca\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbncl\" (UniqueName: \"kubernetes.io/projected/7578c673-39a4-4942-9470-e85a33f20c0e-kube-api-access-xbncl\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-kubernetes\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58595100-3eee-4a74-9476-e81e8609049c-host\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcbd7\" (UniqueName: \"kubernetes.io/projected/75397fac-717f-42a0-a024-70a29cc0357d-kube-api-access-bcbd7\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cni-binary-copy\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104482 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-os-release\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-etc-selinux\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-ovn\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104609 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzqdz\" (UniqueName: \"kubernetes.io/projected/83c21736-acef-47c9-832d-f729ece41b4d-kube-api-access-qzqdz\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-multus\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.107442 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-daemon-config\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-sys-fs\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-kubelet\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-bin\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-kubelet\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-socket-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-modprobe-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-script-lib\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-socket-dir-parent\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-device-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.104997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-netd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105016 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cni-binary-copy\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-config\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-host-slash\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-netns\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.108225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-etc-kubernetes\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.105151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83c21736-acef-47c9-832d-f729ece41b4d-ovn-node-metrics-cert\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106040 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-k8s-cni-cncf-io\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106065 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-cnibin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106095 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-system-cni-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106101 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-conf-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106118 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-run-multus-certs\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-socket-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106321 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-socket-dir-parent\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58595100-3eee-4a74-9476-e81e8609049c-host\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106587 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-kubelet\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-device-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-binary-copy\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106739 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-system-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58595100-3eee-4a74-9476-e81e8609049c-serviceca\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-etc-selinux\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109031 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwc7t\" (UniqueName: \"kubernetes.io/projected/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-kube-api-access-xwc7t\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106842 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-sys-fs\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n2fq\" (UniqueName: \"kubernetes.io/projected/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kube-api-access-8n2fq\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.106992 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-cnibin\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107034 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-multus\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107114 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/75397fac-717f-42a0-a024-70a29cc0357d-os-release\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-dbus\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-tmp\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8pq\" (UniqueName: \"kubernetes.io/projected/58595100-3eee-4a74-9476-e81e8609049c-kube-api-access-nl8pq\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-bin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107472 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-daemon-config\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107530 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.107618 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-host-var-lib-cni-bin\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/75397fac-717f-42a0-a024-70a29cc0357d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-iptables-alerter-script\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7hsk\" (UniqueName: \"kubernetes.io/projected/7c63e06e-a9c8-4cf5-8253-2427e86e6197-kube-api-access-z7hsk\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.109776 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-hostroot\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-registration-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-run\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108907 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-sys\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-systemd-units\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108951 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-netns\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw4rb\" (UniqueName: \"kubernetes.io/projected/270c8cba-3f78-4663-85e1-ba8d92d58158-kube-api-access-qw4rb\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.108998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.109178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-multus-cni-dir\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.109207 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-hostroot\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.109235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-registration-dir\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.110439 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.109240 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-iptables-alerter-script\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.113278 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.113259 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:06.113278 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.113282 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:06.113278 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.113312 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:06.113580 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.113372 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:06.613356434 +0000 UTC m=+3.041244721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:06.113735 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.113716 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:59:06.117491 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.117420 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwc7t\" (UniqueName: \"kubernetes.io/projected/ed7fec0f-84e0-419e-8d60-f32fb07cd23c-kube-api-access-xwc7t\") pod \"multus-qjc6t\" (UID: \"ed7fec0f-84e0-419e-8d60-f32fb07cd23c\") " pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.117779 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.117746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcbd7\" (UniqueName: \"kubernetes.io/projected/75397fac-717f-42a0-a024-70a29cc0357d-kube-api-access-bcbd7\") pod \"multus-additional-cni-plugins-hdhlq\" (UID: \"75397fac-717f-42a0-a024-70a29cc0357d\") " pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.118529 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.118405 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8pq\" (UniqueName: \"kubernetes.io/projected/58595100-3eee-4a74-9476-e81e8609049c-kube-api-access-nl8pq\") pod \"node-ca-655q2\" (UID: \"58595100-3eee-4a74-9476-e81e8609049c\") " pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.119241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.119219 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvps6\" (UniqueName: \"kubernetes.io/projected/e97e504a-4343-4cc4-8ee6-af65a38fb9c6-kube-api-access-jvps6\") pod \"iptables-alerter-prwfw\" (UID: \"e97e504a-4343-4cc4-8ee6-af65a38fb9c6\") " pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.121390 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.121189 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n2fq\" (UniqueName: \"kubernetes.io/projected/58abed4f-6e98-4674-b1f3-6fac6c6e0f47-kube-api-access-8n2fq\") pod \"aws-ebs-csi-driver-node-48zlr\" (UID: \"58abed4f-6e98-4674-b1f3-6fac6c6e0f47\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.209294 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209211 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-conf\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209294 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61016499-c57a-4ff7-8131-62c1ef94de54-agent-certs\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysconfig\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-systemd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209402 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-systemd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysconfig\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209439 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-conf\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209498 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-host\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61016499-c57a-4ff7-8131-62c1ef94de54-konnectivity-ca\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209557 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-host\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbncl\" (UniqueName: \"kubernetes.io/projected/7578c673-39a4-4942-9470-e85a33f20c0e-kube-api-access-xbncl\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-kubernetes\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-ovn\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209610 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzqdz\" (UniqueName: \"kubernetes.io/projected/83c21736-acef-47c9-832d-f729ece41b4d-kube-api-access-qzqdz\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-kubelet\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-bin\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-kubernetes\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-ovn\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.209760 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-modprobe-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-script-lib\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-netd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209885 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-modprobe-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-bin\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209762 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-kubelet\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-config\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209955 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-run-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83c21736-acef-47c9-832d-f729ece41b4d-ovn-node-metrics-cert\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.209998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-dbus\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-tmp\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hsk\" (UniqueName: \"kubernetes.io/projected/7c63e06e-a9c8-4cf5-8253-2427e86e6197-kube-api-access-z7hsk\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61016499-c57a-4ff7-8131-62c1ef94de54-konnectivity-ca\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.210365 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-run\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-sys\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210136 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-dbus\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-systemd-units\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-netns\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw4rb\" (UniqueName: \"kubernetes.io/projected/270c8cba-3f78-4663-85e1-ba8d92d58158-kube-api-access-qw4rb\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-lib-modules\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-var-lib-kubelet\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-log-socket\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-systemd-units\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210373 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-run-netns\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-env-overrides\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-config\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-sys\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-systemd\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-tuned\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210557 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-log-socket\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211156 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.210560 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210592 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-ovnkube-script-lib\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-cni-netd\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.210626 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:06.710609812 +0000 UTC m=+3.138498099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210656 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-var-lib-kubelet\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210755 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-systemd\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-lib-modules\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-node-log\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-run\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210826 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-node-log\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-slash\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-var-lib-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210958 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/83c21736-acef-47c9-832d-f729ece41b4d-env-overrides\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270c8cba-3f78-4663-85e1-ba8d92d58158-hosts-file\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.210995 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-sysctl-d\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-host-slash\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.211976 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211028 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270c8cba-3f78-4663-85e1-ba8d92d58158-hosts-file\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-kubelet-config\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-var-lib-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.211089 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-etc-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-kubelet-config\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.211203 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:06.711186035 +0000 UTC m=+3.139074334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c8cba-3f78-4663-85e1-ba8d92d58158-tmp-dir\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211214 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/83c21736-acef-47c9-832d-f729ece41b4d-etc-openvswitch\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.212909 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.211503 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c8cba-3f78-4663-85e1-ba8d92d58158-tmp-dir\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.213374 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.213337 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-tmp\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.213374 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.213370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c63e06e-a9c8-4cf5-8253-2427e86e6197-etc-tuned\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.213480 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.213379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/83c21736-acef-47c9-832d-f729ece41b4d-ovn-node-metrics-cert\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.213535 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.213512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61016499-c57a-4ff7-8131-62c1ef94de54-agent-certs\") pod \"konnectivity-agent-lvtvj\" (UID: \"61016499-c57a-4ff7-8131-62c1ef94de54\") " pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.222477 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.222455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbncl\" (UniqueName: \"kubernetes.io/projected/7578c673-39a4-4942-9470-e85a33f20c0e-kube-api-access-xbncl\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.224578 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.224556 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzqdz\" (UniqueName: \"kubernetes.io/projected/83c21736-acef-47c9-832d-f729ece41b4d-kube-api-access-qzqdz\") pod \"ovnkube-node-cgfgt\" (UID: \"83c21736-acef-47c9-832d-f729ece41b4d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.224901 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.224883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hsk\" (UniqueName: \"kubernetes.io/projected/7c63e06e-a9c8-4cf5-8253-2427e86e6197-kube-api-access-z7hsk\") pod \"tuned-xz2x8\" (UID: \"7c63e06e-a9c8-4cf5-8253-2427e86e6197\") " pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.224982 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.224924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw4rb\" (UniqueName: \"kubernetes.io/projected/270c8cba-3f78-4663-85e1-ba8d92d58158-kube-api-access-qw4rb\") pod \"node-resolver-rtnkj\" (UID: \"270c8cba-3f78-4663-85e1-ba8d92d58158\") " pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.297333 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.297292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjc6t" Jul 08 18:59:06.305749 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.305731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prwfw" Jul 08 18:59:06.314248 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.314229 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" Jul 08 18:59:06.320823 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.320805 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-655q2" Jul 08 18:59:06.332338 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.332322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" Jul 08 18:59:06.341868 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.341850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" Jul 08 18:59:06.349381 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.349363 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:06.357015 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.357000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rtnkj" Jul 08 18:59:06.362517 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.362495 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:06.713556 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.713521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:06.713780 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.713570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:06.713780 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:06.713594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:06.713780 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713695 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:06.713780 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713772 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:07.713753219 +0000 UTC m=+4.141641518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713695 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713704 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713867 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713842 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:07.713823623 +0000 UTC m=+4.141711911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713886 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:06.713983 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:06.713941 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:07.713929231 +0000 UTC m=+4.141817535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:07.053843 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.053725 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:54:06 +0000 UTC" deadline="2027-12-24 01:56:26.356720043 +0000 UTC" Jul 08 18:59:07.053843 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.053797 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12798h57m19.302926964s" Jul 08 18:59:07.077788 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.077710 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58595100_3eee_4a74_9476_e81e8609049c.slice/crio-3c3a4a2d61149dc9da6b9602724a1afa4dfaedcc58fe15bf7b0aff7da9463403 WatchSource:0}: Error finding container 3c3a4a2d61149dc9da6b9602724a1afa4dfaedcc58fe15bf7b0aff7da9463403: Status 404 returned error can't find the container with id 3c3a4a2d61149dc9da6b9602724a1afa4dfaedcc58fe15bf7b0aff7da9463403 Jul 08 18:59:07.078782 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.078740 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7fec0f_84e0_419e_8d60_f32fb07cd23c.slice/crio-70d62e74e8f71628e2b3848822d1f1fe000035c270534e89dec5fcf916a335d1 WatchSource:0}: Error finding container 70d62e74e8f71628e2b3848822d1f1fe000035c270534e89dec5fcf916a335d1: Status 404 returned error can't find the container with id 70d62e74e8f71628e2b3848822d1f1fe000035c270534e89dec5fcf916a335d1 Jul 08 18:59:07.080684 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.080653 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c63e06e_a9c8_4cf5_8253_2427e86e6197.slice/crio-5ee75a8d4b4a3cf857c8e5a6cda63e7debe2cc8efa4010e661c29205a362713d WatchSource:0}: Error finding container 5ee75a8d4b4a3cf857c8e5a6cda63e7debe2cc8efa4010e661c29205a362713d: Status 404 returned error can't find the container with id 5ee75a8d4b4a3cf857c8e5a6cda63e7debe2cc8efa4010e661c29205a362713d Jul 08 18:59:07.083887 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.083872 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61016499_c57a_4ff7_8131_62c1ef94de54.slice/crio-510531fae2a1e263b5b249c7b2f448e250ee72c8d61ad05fe83766492a0682e9 WatchSource:0}: Error finding container 510531fae2a1e263b5b249c7b2f448e250ee72c8d61ad05fe83766492a0682e9: Status 404 returned error can't find the container with id 510531fae2a1e263b5b249c7b2f448e250ee72c8d61ad05fe83766492a0682e9 Jul 08 18:59:07.084409 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.084375 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58abed4f_6e98_4674_b1f3_6fac6c6e0f47.slice/crio-4e4ed8d890eee1dc18fe64d2ca9b6dd268b673f4665b664be2eb0c37487e0d01 WatchSource:0}: Error finding container 4e4ed8d890eee1dc18fe64d2ca9b6dd268b673f4665b664be2eb0c37487e0d01: Status 404 returned error can't find the container with id 4e4ed8d890eee1dc18fe64d2ca9b6dd268b673f4665b664be2eb0c37487e0d01 Jul 08 18:59:07.085851 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.085830 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97e504a_4343_4cc4_8ee6_af65a38fb9c6.slice/crio-17d7d9f6a7f7ac6d6c7d112376a1ceff25255515d83faa48e4f54cb035c6e261 WatchSource:0}: Error finding container 17d7d9f6a7f7ac6d6c7d112376a1ceff25255515d83faa48e4f54cb035c6e261: Status 404 returned error can't find the container with id 17d7d9f6a7f7ac6d6c7d112376a1ceff25255515d83faa48e4f54cb035c6e261 Jul 08 18:59:07.087013 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.086623 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c21736_acef_47c9_832d_f729ece41b4d.slice/crio-b5e5b0beeed44d38088ba784c0c002dc98c5a509bbfef6d7f8591294927a0ca7 WatchSource:0}: Error finding container b5e5b0beeed44d38088ba784c0c002dc98c5a509bbfef6d7f8591294927a0ca7: Status 404 returned error can't find the container with id b5e5b0beeed44d38088ba784c0c002dc98c5a509bbfef6d7f8591294927a0ca7 Jul 08 18:59:07.087796 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.087724 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270c8cba_3f78_4663_85e1_ba8d92d58158.slice/crio-b9b340e4d11c645672fc68bb986d1467241135b897d6ba6c88b7ae099be09091 WatchSource:0}: Error finding container b9b340e4d11c645672fc68bb986d1467241135b897d6ba6c88b7ae099be09091: Status 404 returned error can't find the container with id b9b340e4d11c645672fc68bb986d1467241135b897d6ba6c88b7ae099be09091 Jul 08 18:59:07.090676 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:07.090600 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75397fac_717f_42a0_a024_70a29cc0357d.slice/crio-1b41953c3928f66c9a971f22d58b7e228294f02a564fea34147437cfe9761d2c WatchSource:0}: Error finding container 1b41953c3928f66c9a971f22d58b7e228294f02a564fea34147437cfe9761d2c: Status 404 returned error can't find the container with id 1b41953c3928f66c9a971f22d58b7e228294f02a564fea34147437cfe9761d2c Jul 08 18:59:07.208789 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.208627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:07.208917 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.208860 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:07.217319 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.217275 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prwfw" event={"ID":"e97e504a-4343-4cc4-8ee6-af65a38fb9c6","Type":"ContainerStarted","Data":"17d7d9f6a7f7ac6d6c7d112376a1ceff25255515d83faa48e4f54cb035c6e261"} Jul 08 18:59:07.218247 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.218226 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-655q2" event={"ID":"58595100-3eee-4a74-9476-e81e8609049c","Type":"ContainerStarted","Data":"3c3a4a2d61149dc9da6b9602724a1afa4dfaedcc58fe15bf7b0aff7da9463403"} Jul 08 18:59:07.219886 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.219865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" event={"ID":"872c3c853569ccf8888d4463b966dc40","Type":"ContainerStarted","Data":"6176a6f6f77502891c61370cac91fc0b9e4372d6f034432edb4e57552cd3eaab"} Jul 08 18:59:07.221205 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.221184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" event={"ID":"58abed4f-6e98-4674-b1f3-6fac6c6e0f47","Type":"ContainerStarted","Data":"4e4ed8d890eee1dc18fe64d2ca9b6dd268b673f4665b664be2eb0c37487e0d01"} Jul 08 18:59:07.222169 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.222139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerStarted","Data":"1b41953c3928f66c9a971f22d58b7e228294f02a564fea34147437cfe9761d2c"} Jul 08 18:59:07.223103 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.223086 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rtnkj" event={"ID":"270c8cba-3f78-4663-85e1-ba8d92d58158","Type":"ContainerStarted","Data":"b9b340e4d11c645672fc68bb986d1467241135b897d6ba6c88b7ae099be09091"} Jul 08 18:59:07.225870 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.225847 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvtvj" event={"ID":"61016499-c57a-4ff7-8131-62c1ef94de54","Type":"ContainerStarted","Data":"510531fae2a1e263b5b249c7b2f448e250ee72c8d61ad05fe83766492a0682e9"} Jul 08 18:59:07.228100 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.228084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjc6t" event={"ID":"ed7fec0f-84e0-419e-8d60-f32fb07cd23c","Type":"ContainerStarted","Data":"70d62e74e8f71628e2b3848822d1f1fe000035c270534e89dec5fcf916a335d1"} Jul 08 18:59:07.229655 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.229635 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" event={"ID":"7c63e06e-a9c8-4cf5-8253-2427e86e6197","Type":"ContainerStarted","Data":"5ee75a8d4b4a3cf857c8e5a6cda63e7debe2cc8efa4010e661c29205a362713d"} Jul 08 18:59:07.231825 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.231808 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"b5e5b0beeed44d38088ba784c0c002dc98c5a509bbfef6d7f8591294927a0ca7"} Jul 08 18:59:07.240891 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.240843 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.ec2.internal" podStartSLOduration=2.240831766 podStartE2EDuration="2.240831766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:07.240157489 +0000 UTC m=+3.668045798" watchObservedRunningTime="2026-07-08 18:59:07.240831766 +0000 UTC m=+3.668720074" Jul 08 18:59:07.720843 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.720763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:07.720843 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.720817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:07.721073 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:07.720851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:07.721073 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.720995 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:07.721073 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721012 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:07.721073 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721034 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:07.721274 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721096 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:09.721078227 +0000 UTC m=+6.148966537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:07.721519 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721494 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:07.721597 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721562 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:09.721544524 +0000 UTC m=+6.149432812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:07.721656 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721626 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:07.721701 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:07.721660 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:09.721648698 +0000 UTC m=+6.149536985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:08.054642 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.054561 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:54:06 +0000 UTC" deadline="2028-03-13 06:45:08.059957202 +0000 UTC" Jul 08 18:59:08.054642 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.054600 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14723h46m0.005361765s" Jul 08 18:59:08.209465 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.209413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:08.209656 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:08.209541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:08.210039 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.210018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:08.210185 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:08.210130 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:08.249181 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.249084 2580 generic.go:356] "Generic (PLEG): container finished" podID="1c32a2d252b8e16f47064d5493ac88b8" containerID="39f62bbc0c4d4cb9cd876d58936104774b3b3a4c1de4f15745dba344023ee8d3" exitCode=0 Jul 08 18:59:08.249361 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:08.249175 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" event={"ID":"1c32a2d252b8e16f47064d5493ac88b8","Type":"ContainerDied","Data":"39f62bbc0c4d4cb9cd876d58936104774b3b3a4c1de4f15745dba344023ee8d3"} Jul 08 18:59:09.207985 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:09.207958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:09.208454 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.208083 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:09.285714 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:09.285668 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" event={"ID":"1c32a2d252b8e16f47064d5493ac88b8","Type":"ContainerStarted","Data":"d2d4bdb9f9b54ba716c5bbf44e91c04d5c09cf1cf054f8a8408906922b53e89d"} Jul 08 18:59:09.738062 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:09.738028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:09.738232 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:09.738074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:09.738232 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:09.738103 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:09.738385 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738246 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:09.738385 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738267 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:09.738385 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738279 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:09.738385 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738355 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:13.738335656 +0000 UTC m=+10.166223966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:09.738785 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738765 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:09.738874 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738820 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:13.738805497 +0000 UTC m=+10.166693798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:09.738934 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738879 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:09.738934 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:09.738912 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:13.738900519 +0000 UTC m=+10.166788805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:10.208704 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:10.208178 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:10.208704 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:10.208326 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:10.208704 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:10.208440 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:10.208704 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:10.208556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:11.208170 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:11.208138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:11.208358 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:11.208273 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:12.210511 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:12.210428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:12.210946 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:12.210530 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:12.210946 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:12.210840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:12.210946 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:12.210895 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:13.208314 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:13.208132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:13.208314 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.208261 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:13.769363 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:13.769327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:13.769375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769487 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:13.769522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769554 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:21.76953252 +0000 UTC m=+18.197420808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769604 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769619 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769628 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769673 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:21.769657227 +0000 UTC m=+18.197545533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769489 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:13.769888 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:13.769711 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:21.769700915 +0000 UTC m=+18.197589216 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:14.209598 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:14.209472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:14.209598 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:14.209551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:14.209805 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:14.209676 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:14.210026 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:14.209982 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:15.208954 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:15.208915 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:15.209439 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:15.209037 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:16.211080 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:16.211053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:16.211549 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:16.211170 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:16.211549 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:16.211053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:16.211668 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:16.211641 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:17.208152 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:17.208116 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:17.208291 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:17.208268 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:18.211883 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:18.211854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:18.212319 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:18.211888 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:18.212319 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:18.211990 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:18.212319 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:18.212119 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:19.207975 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:19.207942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:19.208144 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:19.208110 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:20.208712 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:20.208679 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:20.209105 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:20.208809 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:20.209105 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:20.208865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:20.209105 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:20.208999 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:21.208283 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:21.208243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:21.208515 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.208395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:21.827350 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:21.827313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:21.827361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:21.827390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827483 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827494 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827494 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827547 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 18:59:37.827526487 +0000 UTC m=+34.255414781 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827511 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827566 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:37.827555838 +0000 UTC m=+34.255444133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827569 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:21.827751 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:21.827612 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 18:59:37.827594961 +0000 UTC m=+34.255483261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:22.208463 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:22.208383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:22.208625 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:22.208505 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:22.208625 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:22.208556 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:22.208743 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:22.208650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:23.208385 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:23.208352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:23.208785 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:23.208479 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:24.209399 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:24.209314 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:24.209864 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:24.209427 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:24.209864 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:24.209482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:24.209864 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:24.209606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:25.208079 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:25.208046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:25.208230 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:25.208139 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:26.208871 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.208521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:26.209560 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.208592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:26.209560 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:26.209009 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:26.209560 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:26.209126 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:26.324291 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.324182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjc6t" event={"ID":"ed7fec0f-84e0-419e-8d60-f32fb07cd23c","Type":"ContainerStarted","Data":"e4f01986477b8b8c705c26e6fea1f66a37f5cfa6e92c378d790028688e01eb4d"} Jul 08 18:59:26.325731 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.325707 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" event={"ID":"7c63e06e-a9c8-4cf5-8253-2427e86e6197","Type":"ContainerStarted","Data":"d9f7584892d900f328ddc6f7de7e28f14d32cc3636aeef27c3c64092995b37b0"} Jul 08 18:59:26.328678 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.328661 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 18:59:26.328999 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.328982 2580 generic.go:356] "Generic (PLEG): container finished" podID="83c21736-acef-47c9-832d-f729ece41b4d" containerID="51ca15b05cc35fa94f0dbcf88066fa7f09a4c3c33fe9dcd444a1ed06f75012a7" exitCode=1 Jul 08 18:59:26.329071 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"1602a9e3c5a403c99dd401ec0f18e91307268c627e5d538f5121e2c6fd8d04e4"} Jul 08 18:59:26.329114 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329084 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"121b9812e4ea9de6bfbf6862ab4370bd3a75cc99f8d7c2dbaea18c22a3009719"} Jul 08 18:59:26.329157 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"3cb4f2e3fb85f216d5f9635f4e30bbd9811751ea78cd0db52da0d6857db02d91"} Jul 08 18:59:26.329157 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"41de2accf4f754207ba5ccecac1f8e761c11a946213c94a5fc546ba04c6d6cf3"} Jul 08 18:59:26.329229 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329189 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerDied","Data":"51ca15b05cc35fa94f0dbcf88066fa7f09a4c3c33fe9dcd444a1ed06f75012a7"} Jul 08 18:59:26.329261 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.329233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"8635ad31944897759e04aab3b0aecfa5cdadbd13c8f901e399255d2bf928ff0c"} Jul 08 18:59:26.330619 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.330598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-655q2" event={"ID":"58595100-3eee-4a74-9476-e81e8609049c","Type":"ContainerStarted","Data":"d8602d8b8c0d01c310c839569ee8cbfffb623d0c35f734374db721ff8dbe1ee9"} Jul 08 18:59:26.331906 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.331887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" event={"ID":"58abed4f-6e98-4674-b1f3-6fac6c6e0f47","Type":"ContainerStarted","Data":"fffdfbb66042adaf2d7624845d4f9f8e17cfd8bf404f941fd58e9e6a83673b91"} Jul 08 18:59:26.333250 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.333229 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="db3b9cdc707e438732aab91552e8db286cd168ebc590001b29aae2c4dbb80550" exitCode=0 Jul 08 18:59:26.333383 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.333322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"db3b9cdc707e438732aab91552e8db286cd168ebc590001b29aae2c4dbb80550"} Jul 08 18:59:26.335142 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.335122 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rtnkj" event={"ID":"270c8cba-3f78-4663-85e1-ba8d92d58158","Type":"ContainerStarted","Data":"b331b651cc765c65d3adcaa76bb9480448376af06851852661bef689de5d2a61"} Jul 08 18:59:26.336546 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.336466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvtvj" event={"ID":"61016499-c57a-4ff7-8131-62c1ef94de54","Type":"ContainerStarted","Data":"d5abe822a3829c4ff5b236f8db1220a6cf5adfb8ae2d21351d58d51d54d9007b"} Jul 08 18:59:26.346880 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.346840 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qjc6t" podStartSLOduration=3.900543797 podStartE2EDuration="22.346829197s" totalImagesPullingTime="18.4462854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:26.346731319 +0000 UTC m=+22.774619628" watchObservedRunningTime="2026-07-08 18:59:26.346829197 +0000 UTC m=+22.774717505" Jul 08 18:59:26.347046 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.347028 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.ec2.internal" podStartSLOduration=21.347023951 podStartE2EDuration="21.347023951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:09.303619108 +0000 UTC m=+5.731507417" watchObservedRunningTime="2026-07-08 18:59:26.347023951 +0000 UTC m=+22.774912258" Jul 08 18:59:26.387252 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.387205 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xz2x8" podStartSLOduration=3.957349602 podStartE2EDuration="22.387191352s" totalImagesPullingTime="18.42984175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:26.386810248 +0000 UTC m=+22.814698556" watchObservedRunningTime="2026-07-08 18:59:26.387191352 +0000 UTC m=+22.815079659" Jul 08 18:59:26.407330 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.407267 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rtnkj" podStartSLOduration=3.982355704 podStartE2EDuration="22.4072517s" totalImagesPullingTime="18.424895996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:26.406674318 +0000 UTC m=+22.834562626" watchObservedRunningTime="2026-07-08 18:59:26.4072517 +0000 UTC m=+22.835139985" Jul 08 18:59:26.423862 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.423820 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-655q2" podStartSLOduration=3.991424233 podStartE2EDuration="22.423807094s" totalImagesPullingTime="18.432382861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:26.423589915 +0000 UTC m=+22.851478223" watchObservedRunningTime="2026-07-08 18:59:26.423807094 +0000 UTC m=+22.851695402" Jul 08 18:59:26.440492 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.440447 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lvtvj" podStartSLOduration=4.013638263 podStartE2EDuration="22.440434575s" totalImagesPullingTime="18.426796312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:26.4403221 +0000 UTC m=+22.868210406" watchObservedRunningTime="2026-07-08 18:59:26.440434575 +0000 UTC m=+22.868322880" Jul 08 18:59:26.920437 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:26.920412 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:59:27.070973 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.070835 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:59:26.920432446Z","UUID":"adab8177-1ec1-4d28-a3bb-4ebeb908bac4","Handler":null,"Name":"","Endpoint":""} Jul 08 18:59:27.072339 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.072317 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:59:27.072450 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.072362 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:59:27.208582 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.208554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:27.208713 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:27.208652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:27.321604 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.321522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:27.322040 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.321878 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:27.342096 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.342066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" event={"ID":"58abed4f-6e98-4674-b1f3-6fac6c6e0f47","Type":"ContainerStarted","Data":"48d490c0ae03a4b077bfe562634264c8d593e8236360e551022aa7eaf9fd568c"} Jul 08 18:59:27.343971 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.343906 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prwfw" event={"ID":"e97e504a-4343-4cc4-8ee6-af65a38fb9c6","Type":"ContainerStarted","Data":"3174344eb8014823cfa47e9f64bb11e638fc3ec58716990e5c8253bc2d9d7d96"} Jul 08 18:59:27.345455 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.345435 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:27.345567 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.345491 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lvtvj" Jul 08 18:59:27.364884 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:27.364838 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-prwfw" podStartSLOduration=4.940504537 podStartE2EDuration="23.364822725s" totalImagesPullingTime="18.424318188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:27.3646959 +0000 UTC m=+23.792584210" watchObservedRunningTime="2026-07-08 18:59:27.364822725 +0000 UTC m=+23.792711035" Jul 08 18:59:28.208808 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:28.208732 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:28.208967 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:28.208742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:28.208967 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:28.208855 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:28.208967 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:28.208942 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:28.348188 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:28.348158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" event={"ID":"58abed4f-6e98-4674-b1f3-6fac6c6e0f47","Type":"ContainerStarted","Data":"fae187d2d989d91b35b18acd562e0ca181e244235e82d68e1e2fdb05ce853f9b"} Jul 08 18:59:28.374040 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:28.373842 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48zlr" podStartSLOduration=3.93871555 podStartE2EDuration="24.37382377s" totalImagesPullingTime="20.43510822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:28.373265522 +0000 UTC m=+24.801153832" watchObservedRunningTime="2026-07-08 18:59:28.37382377 +0000 UTC m=+24.801712087" Jul 08 18:59:29.208752 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:29.208726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:29.208952 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:29.208830 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:29.356241 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:29.356194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 18:59:29.356643 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:29.356559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"bbfb02e867f60a3f0eb2e0cd7a6e6670aaa4f6a1c8a0a484a52779ebb7f73da7"} Jul 08 18:59:30.208431 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:30.208398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:30.208607 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:30.208398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:30.208607 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:30.208529 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:30.208607 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:30.208592 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:31.208014 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.207856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:31.208579 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:31.208112 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:31.364718 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.364695 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 18:59:31.365103 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.365078 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"a1da50ddd00fa9cf5b61ffe97baa7211694756ba88a8e047ddc171885083dd4a"} Jul 08 18:59:31.365454 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.365290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:31.365454 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.365379 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:31.365454 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.365426 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:31.365619 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.365507 2580 scope.go:117] "RemoveContainer" containerID="51ca15b05cc35fa94f0dbcf88066fa7f09a4c3c33fe9dcd444a1ed06f75012a7" Jul 08 18:59:31.367154 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.367126 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="af75dad143c385107f82d96261c4d329eedf7fb6b09c3f99f72315b370e1e35e" exitCode=0 Jul 08 18:59:31.367280 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.367171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"af75dad143c385107f82d96261c4d329eedf7fb6b09c3f99f72315b370e1e35e"} Jul 08 18:59:31.380981 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.380962 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:31.381267 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:31.381254 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 18:59:32.211642 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:32.209747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:32.211642 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:32.209866 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:32.211642 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:32.210266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:32.211642 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:32.210377 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:32.372576 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:32.372546 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 18:59:32.372894 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:32.372872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" event={"ID":"83c21736-acef-47c9-832d-f729ece41b4d","Type":"ContainerStarted","Data":"e06b56f7a3f7a5650117fe7f96a24428a79b74534d9514f244aab60502185445"} Jul 08 18:59:32.402532 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:32.402487 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" podStartSLOduration=9.812810542 podStartE2EDuration="28.402471755s" totalImagesPullingTime="18.589661213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:32.40120351 +0000 UTC m=+28.829091829" watchObservedRunningTime="2026-07-08 18:59:32.402471755 +0000 UTC m=+28.830360044" Jul 08 18:59:33.176005 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.175743 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gmnq"] Jul 08 18:59:33.176225 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.176144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:33.176295 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:33.176245 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:33.176467 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.176430 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hclm2"] Jul 08 18:59:33.176599 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.176545 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:33.176679 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:33.176648 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:33.177041 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.177019 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ljgz5"] Jul 08 18:59:33.177126 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.177106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:33.177216 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:33.177196 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:33.377741 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.377672 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="47a9a784c29864629c847c5153de853ad19e31e75acc010716e16071c9a86cc6" exitCode=0 Jul 08 18:59:33.378077 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:33.377752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"47a9a784c29864629c847c5153de853ad19e31e75acc010716e16071c9a86cc6"} Jul 08 18:59:35.208653 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:35.208624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:35.209284 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:35.208680 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:35.209284 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:35.208761 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:35.209284 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:35.208817 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:35.209284 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:35.208839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:35.209284 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:35.208943 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:35.384645 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:35.384565 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="3d2c8510fa2f7084adc5a4fff58060449560308c980802b4203e2a8e8028d7ae" exitCode=0 Jul 08 18:59:35.384645 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:35.384615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"3d2c8510fa2f7084adc5a4fff58060449560308c980802b4203e2a8e8028d7ae"} Jul 08 18:59:37.208911 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.208820 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:37.208911 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.208865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:37.208911 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.208865 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:37.209474 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.208954 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 18:59:37.209474 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.209073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ljgz5" podUID="aafbf116-efdf-4485-a2d0-c79803816965" Jul 08 18:59:37.209474 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.209157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hclm2" podUID="ffea4bdc-91db-45ba-9dee-f71d26b67dbd" Jul 08 18:59:37.845681 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.845647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.845731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:37.845762 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845827 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845860 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845829 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845915 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845897 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 19:00:09.845876275 +0000 UTC m=+66.273764563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:59:37.845939 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845928 2580 projected.go:196] Error preparing data for projected volume kube-api-access-997wp for pod openshift-network-diagnostics/network-check-target-ljgz5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:37.846260 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845959 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret podName:ffea4bdc-91db-45ba-9dee-f71d26b67dbd nodeName:}" failed. No retries permitted until 2026-07-08 19:00:09.84593425 +0000 UTC m=+66.273822553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret") pod "global-pull-secret-syncer-hclm2" (UID: "ffea4bdc-91db-45ba-9dee-f71d26b67dbd") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:59:37.846260 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:37.845977 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp podName:aafbf116-efdf-4485-a2d0-c79803816965 nodeName:}" failed. No retries permitted until 2026-07-08 19:00:09.845967139 +0000 UTC m=+66.273855439 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-997wp" (UniqueName: "kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp") pod "network-check-target-ljgz5" (UID: "aafbf116-efdf-4485-a2d0-c79803816965") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:59:38.314948 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.314715 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-223.ec2.internal" event="NodeReady" Jul 08 18:59:38.315376 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.315022 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:59:38.361814 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.361786 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vdx2v"] Jul 08 18:59:38.364713 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.364612 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.364900 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.364883 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n9mqf"] Jul 08 18:59:38.366719 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.366478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.367206 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.367184 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:59:38.367206 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.367196 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:59:38.367483 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.367469 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-97fdz\"" Jul 08 18:59:38.368999 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.368975 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:59:38.369109 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.369038 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:59:38.369221 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.369204 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:59:38.370080 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.370059 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hs7p8\"" Jul 08 18:59:38.379432 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.379408 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdx2v"] Jul 08 18:59:38.380229 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.380181 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n9mqf"] Jul 08 18:59:38.449510 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms9sr\" (UniqueName: \"kubernetes.io/projected/a5a50da0-90dd-407a-a1bf-b57bd756124e-kube-api-access-ms9sr\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.449636 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.449636 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a50da0-90dd-407a-a1bf-b57bd756124e-tmp-dir\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.449745 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ptl\" (UniqueName: \"kubernetes.io/projected/b4a10718-e056-4a7a-851b-be327aff9606-kube-api-access-f6ptl\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.449745 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4a10718-e056-4a7a-851b-be327aff9606-cert\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.449745 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.449705 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a50da0-90dd-407a-a1bf-b57bd756124e-config-volume\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.550863 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a50da0-90dd-407a-a1bf-b57bd756124e-tmp-dir\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.550863 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ptl\" (UniqueName: \"kubernetes.io/projected/b4a10718-e056-4a7a-851b-be327aff9606-kube-api-access-f6ptl\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.551087 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4a10718-e056-4a7a-851b-be327aff9606-cert\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.551087 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a50da0-90dd-407a-a1bf-b57bd756124e-config-volume\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.551087 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms9sr\" (UniqueName: \"kubernetes.io/projected/a5a50da0-90dd-407a-a1bf-b57bd756124e-kube-api-access-ms9sr\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.551087 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.550985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.551273 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:38.551091 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:38.551273 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:38.551156 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:39.05113749 +0000 UTC m=+35.479025793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 18:59:38.551550 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.551522 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a50da0-90dd-407a-a1bf-b57bd756124e-tmp-dir\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.551726 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.551682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5a50da0-90dd-407a-a1bf-b57bd756124e-config-volume\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.555803 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.555783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4a10718-e056-4a7a-851b-be327aff9606-cert\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.562331 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.562290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms9sr\" (UniqueName: \"kubernetes.io/projected/a5a50da0-90dd-407a-a1bf-b57bd756124e-kube-api-access-ms9sr\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:38.562771 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.562750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ptl\" (UniqueName: \"kubernetes.io/projected/b4a10718-e056-4a7a-851b-be327aff9606-kube-api-access-f6ptl\") pod \"ingress-canary-n9mqf\" (UID: \"b4a10718-e056-4a7a-851b-be327aff9606\") " pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:38.686980 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:38.686881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n9mqf" Jul 08 18:59:39.054814 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.054738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:39.054970 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:39.054891 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:39.054970 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:39.054967 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:40.054949331 +0000 UTC m=+36.482837621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 18:59:39.208173 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.208142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 18:59:39.208353 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.208142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 18:59:39.208353 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.208144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 18:59:39.212587 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212568 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:59:39.212711 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212617 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:59:39.212711 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212687 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7bzch\"" Jul 08 18:59:39.212711 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212704 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:59:39.212848 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212814 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:59:39.212883 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.212845 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gvgdf\"" Jul 08 18:59:39.546485 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:39.546455 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n9mqf"] Jul 08 18:59:39.552255 ip-10-0-142-223 kubenswrapper[2580]: W0708 18:59:39.552230 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a10718_e056_4a7a_851b_be327aff9606.slice/crio-468a5a49b6edf18a6e8a7b50afea3841b20c9de2b827573ab760840d791666fd WatchSource:0}: Error finding container 468a5a49b6edf18a6e8a7b50afea3841b20c9de2b827573ab760840d791666fd: Status 404 returned error can't find the container with id 468a5a49b6edf18a6e8a7b50afea3841b20c9de2b827573ab760840d791666fd Jul 08 18:59:40.061553 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:40.061516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:40.061738 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:40.061685 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:40.061803 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:40.061752 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:42.061735305 +0000 UTC m=+38.489623595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 18:59:40.400132 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:40.400057 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="6ebf938077926b685591b72cd7f135b5e71c1d6f81fc6677d45dc659495f3d88" exitCode=0 Jul 08 18:59:40.400316 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:40.400146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"6ebf938077926b685591b72cd7f135b5e71c1d6f81fc6677d45dc659495f3d88"} Jul 08 18:59:40.401574 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:40.401548 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n9mqf" event={"ID":"b4a10718-e056-4a7a-851b-be327aff9606","Type":"ContainerStarted","Data":"468a5a49b6edf18a6e8a7b50afea3841b20c9de2b827573ab760840d791666fd"} Jul 08 18:59:41.407389 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:41.407353 2580 generic.go:356] "Generic (PLEG): container finished" podID="75397fac-717f-42a0-a024-70a29cc0357d" containerID="cb6a0b5acdd89302e9a10260034bfe3f1e15a2731aeecd802ae71ad06ba24f3f" exitCode=0 Jul 08 18:59:41.407838 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:41.407417 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerDied","Data":"cb6a0b5acdd89302e9a10260034bfe3f1e15a2731aeecd802ae71ad06ba24f3f"} Jul 08 18:59:42.078160 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:42.078081 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:42.078295 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:42.078238 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:42.078363 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:42.078325 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:46.078288833 +0000 UTC m=+42.506177122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 18:59:42.412972 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:42.412884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n9mqf" event={"ID":"b4a10718-e056-4a7a-851b-be327aff9606","Type":"ContainerStarted","Data":"241941bbca85b3fb6727968e0bc10b0f3061c724eeb77b5443ee1743969fbe4a"} Jul 08 18:59:42.416036 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:42.416011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" event={"ID":"75397fac-717f-42a0-a024-70a29cc0357d","Type":"ContainerStarted","Data":"e2a845d4df2fd6662b7bb3d326a41d898c3fde456b1a49793930aad510094589"} Jul 08 18:59:42.438765 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:42.438722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n9mqf" podStartSLOduration=2.209331001 podStartE2EDuration="4.438707669s" totalImagesPullingTime="2.229376668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:42.43823466 +0000 UTC m=+38.866122969" watchObservedRunningTime="2026-07-08 18:59:42.438707669 +0000 UTC m=+38.866595954" Jul 08 18:59:42.474963 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:42.474847 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hdhlq" podStartSLOduration=8.914513127 podStartE2EDuration="38.47482588s" totalImagesPullingTime="29.560312753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:42.473524227 +0000 UTC m=+38.901412535" watchObservedRunningTime="2026-07-08 18:59:42.47482588 +0000 UTC m=+38.902714188" Jul 08 18:59:46.109970 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:46.109930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:46.110336 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:46.110079 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:46.110336 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:46.110140 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 18:59:54.110123461 +0000 UTC m=+50.538011973 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 18:59:54.165524 ip-10-0-142-223 kubenswrapper[2580]: I0708 18:59:54.165483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 18:59:54.166008 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:54.165587 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:59:54.166008 ip-10-0-142-223 kubenswrapper[2580]: E0708 18:59:54.165643 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 19:00:10.165631599 +0000 UTC m=+66.593519885 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 19:00:02.816885 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.816851 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz"] Jul 08 19:00:02.818999 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.818982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:02.823192 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.823162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 19:00:02.823347 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.823162 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 19:00:02.823424 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.823347 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 19:00:02.823424 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.823351 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 19:00:02.823424 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.823385 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4gxf9\"" Jul 08 19:00:02.833046 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.833026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz"] Jul 08 19:00:02.925234 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.925210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7kv4\" (UniqueName: \"kubernetes.io/projected/18093611-d423-4bb2-8c69-e7d8744b2928-kube-api-access-q7kv4\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:02.925376 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:02.925266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18093611-d423-4bb2-8c69-e7d8744b2928-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.025912 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.025892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7kv4\" (UniqueName: \"kubernetes.io/projected/18093611-d423-4bb2-8c69-e7d8744b2928-kube-api-access-q7kv4\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.025987 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.025940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18093611-d423-4bb2-8c69-e7d8744b2928-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.029689 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.029664 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18093611-d423-4bb2-8c69-e7d8744b2928-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.036331 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.036312 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7kv4\" (UniqueName: \"kubernetes.io/projected/18093611-d423-4bb2-8c69-e7d8744b2928-kube-api-access-q7kv4\") pod \"managed-serviceaccount-addon-agent-d74998666-2r4mz\" (UID: \"18093611-d423-4bb2-8c69-e7d8744b2928\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.137125 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.137077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" Jul 08 19:00:03.257348 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.257319 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz"] Jul 08 19:00:03.261035 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:00:03.261007 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18093611_d423_4bb2_8c69_e7d8744b2928.slice/crio-7f8dbb01cffbf4edbf12059364ee8caee29bde2cd771aa327ef624b0724c83a8 WatchSource:0}: Error finding container 7f8dbb01cffbf4edbf12059364ee8caee29bde2cd771aa327ef624b0724c83a8: Status 404 returned error can't find the container with id 7f8dbb01cffbf4edbf12059364ee8caee29bde2cd771aa327ef624b0724c83a8 Jul 08 19:00:03.387808 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.387739 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgfgt" Jul 08 19:00:03.462493 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:03.462464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" event={"ID":"18093611-d423-4bb2-8c69-e7d8744b2928","Type":"ContainerStarted","Data":"7f8dbb01cffbf4edbf12059364ee8caee29bde2cd771aa327ef624b0724c83a8"} Jul 08 19:00:09.477170 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.477131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" event={"ID":"18093611-d423-4bb2-8c69-e7d8744b2928","Type":"ContainerStarted","Data":"04b60ed4874a128c021e60a6a02e1c820cb1bedc235fd822729f9e5e77f08592"} Jul 08 19:00:09.495918 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.495871 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" podStartSLOduration=1.5606416539999999 podStartE2EDuration="7.495856729s" totalImagesPullingTime="5.935215075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:00:09.494743956 +0000 UTC m=+65.922632268" watchObservedRunningTime="2026-07-08 19:00:09.495856729 +0000 UTC m=+65.923745037" Jul 08 19:00:09.872246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.872139 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:00:09.872246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.872193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 19:00:09.872246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.872216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 19:00:09.875333 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.875296 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:00:09.875387 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.875319 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:00:09.875387 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.875320 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:00:09.882711 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:09.882694 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:00:09.882777 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:09.882767 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 19:01:13.882751058 +0000 UTC m=+130.310639347 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : secret "metrics-daemon-secret" not found Jul 08 19:00:09.884741 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.884717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffea4bdc-91db-45ba-9dee-f71d26b67dbd-original-pull-secret\") pod \"global-pull-secret-syncer-hclm2\" (UID: \"ffea4bdc-91db-45ba-9dee-f71d26b67dbd\") " pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 19:00:09.885110 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.885097 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:00:09.894922 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:09.894896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997wp\" (UniqueName: \"kubernetes.io/projected/aafbf116-efdf-4485-a2d0-c79803816965-kube-api-access-997wp\") pod \"network-check-target-ljgz5\" (UID: \"aafbf116-efdf-4485-a2d0-c79803816965\") " pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 19:00:10.119223 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.119174 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hclm2" Jul 08 19:00:10.135336 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.135295 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7bzch\"" Jul 08 19:00:10.142457 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.142433 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 19:00:10.175062 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.175028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 19:00:10.175218 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:10.175198 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:00:10.175312 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:10.175290 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 19:00:42.1752656 +0000 UTC m=+98.603153888 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 19:00:10.252062 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.252032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hclm2"] Jul 08 19:00:10.255603 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:00:10.255577 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffea4bdc_91db_45ba_9dee_f71d26b67dbd.slice/crio-ba86d693b8d32e66346085745a9a1cecf90ff45a23ec51860ae88015e41fcd2a WatchSource:0}: Error finding container ba86d693b8d32e66346085745a9a1cecf90ff45a23ec51860ae88015e41fcd2a: Status 404 returned error can't find the container with id ba86d693b8d32e66346085745a9a1cecf90ff45a23ec51860ae88015e41fcd2a Jul 08 19:00:10.276915 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.276885 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ljgz5"] Jul 08 19:00:10.281646 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:00:10.281615 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafbf116_efdf_4485_a2d0_c79803816965.slice/crio-875c281a44463c2be72e079793e33a3944b67c374a27af375e40bd2fa18749f9 WatchSource:0}: Error finding container 875c281a44463c2be72e079793e33a3944b67c374a27af375e40bd2fa18749f9: Status 404 returned error can't find the container with id 875c281a44463c2be72e079793e33a3944b67c374a27af375e40bd2fa18749f9 Jul 08 19:00:10.480615 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.480577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ljgz5" event={"ID":"aafbf116-efdf-4485-a2d0-c79803816965","Type":"ContainerStarted","Data":"875c281a44463c2be72e079793e33a3944b67c374a27af375e40bd2fa18749f9"} Jul 08 19:00:10.481678 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:10.481656 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hclm2" event={"ID":"ffea4bdc-91db-45ba-9dee-f71d26b67dbd","Type":"ContainerStarted","Data":"ba86d693b8d32e66346085745a9a1cecf90ff45a23ec51860ae88015e41fcd2a"} Jul 08 19:00:15.497576 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:15.497537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hclm2" event={"ID":"ffea4bdc-91db-45ba-9dee-f71d26b67dbd","Type":"ContainerStarted","Data":"9a576bbd75ced7ad2ff438dd0d2a6a52473addf8418b151c01af4dab115b9621"} Jul 08 19:00:15.498953 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:15.498930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ljgz5" event={"ID":"aafbf116-efdf-4485-a2d0-c79803816965","Type":"ContainerStarted","Data":"d2c9123f9d9f56c74a3362c54503d1190e68becb372146c35a54cc8b5a0430d4"} Jul 08 19:00:15.499063 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:15.499050 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 19:00:15.516110 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:15.516066 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hclm2" podStartSLOduration=66.965901104 podStartE2EDuration="1m11.516050677s" totalImagesPullingTime="4.550149573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:00:15.515830954 +0000 UTC m=+71.943719261" watchObservedRunningTime="2026-07-08 19:00:15.516050677 +0000 UTC m=+71.943938986" Jul 08 19:00:15.534641 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:15.534599 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ljgz5" podStartSLOduration=67.014253915 podStartE2EDuration="1m11.53458554s" totalImagesPullingTime="4.520331625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:00:15.534231047 +0000 UTC m=+71.962119356" watchObservedRunningTime="2026-07-08 19:00:15.53458554 +0000 UTC m=+71.962473848" Jul 08 19:00:42.191344 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:42.191286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 19:00:42.191935 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:42.191466 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:00:42.191935 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:00:42.191551 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 19:01:46.191530591 +0000 UTC m=+162.619418882 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 19:00:46.504447 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:00:46.504412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ljgz5" Jul 08 19:01:13.911804 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:01:13.911750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:01:13.912345 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:13.911871 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:01:13.912345 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:13.911937 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 19:03:15.91192145 +0000 UTC m=+252.339809737 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : secret "metrics-daemon-secret" not found Jul 08 19:01:41.380822 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:41.380783 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vdx2v" podUID="a5a50da0-90dd-407a-a1bf-b57bd756124e" Jul 08 19:01:41.701410 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:01:41.701334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdx2v" Jul 08 19:01:42.226262 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:42.226220 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 19:01:46.223498 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:01:46.223461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 19:01:46.223949 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:46.223651 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:01:46.223949 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:01:46.223718 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls podName:a5a50da0-90dd-407a-a1bf-b57bd756124e nodeName:}" failed. No retries permitted until 2026-07-08 19:03:48.223699382 +0000 UTC m=+284.651587671 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls") pod "dns-default-vdx2v" (UID: "a5a50da0-90dd-407a-a1bf-b57bd756124e") : secret "dns-default-metrics-tls" not found Jul 08 19:01:54.209501 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:01:54.209472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:02:09.767914 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:02:09.767879 2580 generic.go:356] "Generic (PLEG): container finished" podID="18093611-d423-4bb2-8c69-e7d8744b2928" containerID="04b60ed4874a128c021e60a6a02e1c820cb1bedc235fd822729f9e5e77f08592" exitCode=255 Jul 08 19:02:09.768336 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:02:09.767938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" event={"ID":"18093611-d423-4bb2-8c69-e7d8744b2928","Type":"ContainerDied","Data":"04b60ed4874a128c021e60a6a02e1c820cb1bedc235fd822729f9e5e77f08592"} Jul 08 19:02:09.773406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:02:09.773386 2580 scope.go:117] "RemoveContainer" containerID="04b60ed4874a128c021e60a6a02e1c820cb1bedc235fd822729f9e5e77f08592" Jul 08 19:02:10.776405 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:02:10.776364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-d74998666-2r4mz" event={"ID":"18093611-d423-4bb2-8c69-e7d8744b2928","Type":"ContainerStarted","Data":"f78386246286d2b827063d20b4deaa6f1ac83dae1e64d06a3197d1312da798d0"} Jul 08 19:03:15.912916 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:15.912875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:03:15.913425 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:15.912994 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:03:15.913425 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:15.913056 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs podName:7578c673-39a4-4942-9470-e85a33f20c0e nodeName:}" failed. No retries permitted until 2026-07-08 19:05:17.91304188 +0000 UTC m=+374.340930166 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs") pod "network-metrics-daemon-9gmnq" (UID: "7578c673-39a4-4942-9470-e85a33f20c0e") : secret "metrics-daemon-secret" not found Jul 08 19:03:19.733439 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.733402 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8"] Jul 08 19:03:19.734988 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.734972 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl"] Jul 08 19:03:19.735099 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.735080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.736575 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.736552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.737325 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.737284 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9"] Jul 08 19:03:19.737615 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.737593 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 08 19:03:19.737895 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.737874 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 08 19:03:19.738001 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.737982 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 08 19:03:19.738189 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.738176 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:03:19.738239 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.738221 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-57sjj\"" Jul 08 19:03:19.739004 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.738985 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 19:03:19.739352 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.739338 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 19:03:19.739428 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.739392 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5w2fc\"" Jul 08 19:03:19.739528 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.739514 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.739672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.739657 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 19:03:19.739734 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.739718 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:03:19.742935 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.742915 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 08 19:03:19.743960 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.743942 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 08 19:03:19.744176 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.744161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9g4xq\"" Jul 08 19:03:19.744230 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.744217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:03:19.751906 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.751888 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8"] Jul 08 19:03:19.759050 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.759029 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl"] Jul 08 19:03:19.762664 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.762644 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9"] Jul 08 19:03:19.807959 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.807933 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld"] Jul 08 19:03:19.809820 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.809806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" Jul 08 19:03:19.812193 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.812174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-v2trb\"" Jul 08 19:03:19.818724 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.818705 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cf956c5dc-r496w"] Jul 08 19:03:19.820379 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.820360 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.820689 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.820668 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n"] Jul 08 19:03:19.822924 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.822906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:19.823783 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.823768 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 19:03:19.824190 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.824175 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:03:19.824402 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.824387 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sc6h5\"" Jul 08 19:03:19.827202 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.827184 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 19:03:19.830080 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.830054 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 19:03:19.831661 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.831641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld"] Jul 08 19:03:19.832919 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.832898 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-config\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a86510c6-35a5-48f4-badb-aea0921624d9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kj2q\" (UniqueName: \"kubernetes.io/projected/a86510c6-35a5-48f4-badb-aea0921624d9-kube-api-access-6kj2q\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgmv2\" (UniqueName: \"kubernetes.io/projected/d08b88c4-dfe9-4a84-8911-86761acc6f9f-kube-api-access-xgmv2\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.835698 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a86510c6-35a5-48f4-badb-aea0921624d9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.836029 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv6xm\" (UniqueName: \"kubernetes.io/projected/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-kube-api-access-sv6xm\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.836029 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.835755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.838978 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.838959 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 19:03:19.839691 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.839678 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:03:19.840115 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.840095 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:03:19.840208 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.840095 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-p7nzv\"" Jul 08 19:03:19.841342 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.841325 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:03:19.841474 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.841428 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 19:03:19.841556 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.841486 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n"] Jul 08 19:03:19.842807 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.842779 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cf956c5dc-r496w"] Jul 08 19:03:19.910553 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.910528 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:03:19.912246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.912231 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:19.915169 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.915150 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:03:19.915272 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.915185 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:03:19.915481 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.915465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:03:19.915669 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.915657 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2n9mw\"" Jul 08 19:03:19.922754 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.922738 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:03:19.933264 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.933246 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:03:19.936488 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a86510c6-35a5-48f4-badb-aea0921624d9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.936592 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kj2q\" (UniqueName: \"kubernetes.io/projected/a86510c6-35a5-48f4-badb-aea0921624d9-kube-api-access-6kj2q\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.936592 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgmv2\" (UniqueName: \"kubernetes.io/projected/d08b88c4-dfe9-4a84-8911-86761acc6f9f-kube-api-access-xgmv2\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.936592 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gks\" (UniqueName: \"kubernetes.io/projected/125a062b-4f51-4040-bb38-f0a8a610d369-kube-api-access-n2gks\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.936723 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbqdz\" (UniqueName: \"kubernetes.io/projected/8d0494fa-e61e-45a7-b75b-6e9113bb91de-kube-api-access-lbqdz\") pod \"network-check-source-989f8b8dc-t6rld\" (UID: \"8d0494fa-e61e-45a7-b75b-6e9113bb91de\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" Jul 08 19:03:19.936774 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:19.936823 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.936875 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a86510c6-35a5-48f4-badb-aea0921624d9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.936875 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv6xm\" (UniqueName: \"kubernetes.io/projected/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-kube-api-access-sv6xm\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.936875 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:19.936867 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 19:03:19.937019 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-stats-auth\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.937019 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:19.936928 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls podName:d08b88c4-dfe9-4a84-8911-86761acc6f9f nodeName:}" failed. No retries permitted until 2026-07-08 19:03:20.436909682 +0000 UTC m=+256.864797969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qk8d9" (UID: "d08b88c4-dfe9-4a84-8911-86761acc6f9f") : secret "samples-operator-tls" not found Jul 08 19:03:19.937019 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936955 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a86510c6-35a5-48f4-badb-aea0921624d9-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.937019 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.936992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-default-certificate\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh8h7\" (UniqueName: \"kubernetes.io/projected/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-kube-api-access-sh8h7\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937185 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-config\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.937227 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:19.937672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.937657 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-config\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.938921 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.938895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a86510c6-35a5-48f4-badb-aea0921624d9-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:19.939029 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.939012 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.947638 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.947617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv6xm\" (UniqueName: \"kubernetes.io/projected/0b75b4e4-3ce0-418d-90ba-fd705325ff1f-kube-api-access-sv6xm\") pod \"service-ca-operator-5d46d9ffd4-jzrb8\" (UID: \"0b75b4e4-3ce0-418d-90ba-fd705325ff1f\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:19.948083 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.948059 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgmv2\" (UniqueName: \"kubernetes.io/projected/d08b88c4-dfe9-4a84-8911-86761acc6f9f-kube-api-access-xgmv2\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:19.948151 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:19.948090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kj2q\" (UniqueName: \"kubernetes.io/projected/a86510c6-35a5-48f4-badb-aea0921624d9-kube-api-access-6kj2q\") pod \"kube-storage-version-migrator-operator-846bf668f7-llkzl\" (UID: \"a86510c6-35a5-48f4-badb-aea0921624d9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:20.037748 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-default-certificate\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.037748 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.037748 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh8h7\" (UniqueName: \"kubernetes.io/projected/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-kube-api-access-sh8h7\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.037967 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-stats-auth\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.037967 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgnb6\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.037967 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gks\" (UniqueName: \"kubernetes.io/projected/125a062b-4f51-4040-bb38-f0a8a610d369-kube-api-access-n2gks\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.037967 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbqdz\" (UniqueName: \"kubernetes.io/projected/8d0494fa-e61e-45a7-b75b-6e9113bb91de-kube-api-access-lbqdz\") pod \"network-check-source-989f8b8dc-t6rld\" (UID: \"8d0494fa-e61e-45a7-b75b-6e9113bb91de\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" Jul 08 19:03:20.037967 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.037904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.038184 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038184 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038184 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.038184 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.038217 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.038272 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:20.538254552 +0000 UTC m=+256.966142845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.038352 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:20.538335497 +0000 UTC m=+256.966223786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.038417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.038820 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.038503 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:20.038820 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.038552 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:20.538538351 +0000 UTC m=+256.966426638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:20.038820 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.038597 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.040185 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.040167 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-default-certificate\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.040279 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.040221 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-stats-auth\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.045006 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.044987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" Jul 08 19:03:20.046868 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.046847 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh8h7\" (UniqueName: \"kubernetes.io/projected/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-kube-api-access-sh8h7\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.047871 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.047855 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbqdz\" (UniqueName: \"kubernetes.io/projected/8d0494fa-e61e-45a7-b75b-6e9113bb91de-kube-api-access-lbqdz\") pod \"network-check-source-989f8b8dc-t6rld\" (UID: \"8d0494fa-e61e-45a7-b75b-6e9113bb91de\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" Jul 08 19:03:20.047964 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.047949 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gks\" (UniqueName: \"kubernetes.io/projected/125a062b-4f51-4040-bb38-f0a8a610d369-kube-api-access-n2gks\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.051588 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.051568 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" Jul 08 19:03:20.118260 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.118223 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" Jul 08 19:03:20.139125 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.138983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgnb6\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139125 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139125 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.139908 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.139557 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:20.139908 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.139579 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:20.140076 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.139925 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:20.639903543 +0000 UTC m=+257.067791843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:20.140076 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.139945 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.140347 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.140235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.141559 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.141515 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.141812 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.141795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.142890 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.142870 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.151256 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.151213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.151256 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.151267 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgnb6\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.185817 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.185779 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8"] Jul 08 19:03:20.189284 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:20.189255 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b75b4e4_3ce0_418d_90ba_fd705325ff1f.slice/crio-9264982c4dad0eb3849583154eaf8cd9236f44bd483d684efce913a6dbcc25d6 WatchSource:0}: Error finding container 9264982c4dad0eb3849583154eaf8cd9236f44bd483d684efce913a6dbcc25d6: Status 404 returned error can't find the container with id 9264982c4dad0eb3849583154eaf8cd9236f44bd483d684efce913a6dbcc25d6 Jul 08 19:03:20.204670 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.204643 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl"] Jul 08 19:03:20.206800 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:20.206778 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86510c6_35a5_48f4_badb_aea0921624d9.slice/crio-f355b0e5d872cb9641c0bd0ad6b72347fd66021d95fb696242d10608c3031e69 WatchSource:0}: Error finding container f355b0e5d872cb9641c0bd0ad6b72347fd66021d95fb696242d10608c3031e69: Status 404 returned error can't find the container with id f355b0e5d872cb9641c0bd0ad6b72347fd66021d95fb696242d10608c3031e69 Jul 08 19:03:20.249259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.249227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld"] Jul 08 19:03:20.251929 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:20.251906 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0494fa_e61e_45a7_b75b_6e9113bb91de.slice/crio-c529bb60a2d1d5cb5e736d682e7716320ed37be034d4c855595777174e5ada9a WatchSource:0}: Error finding container c529bb60a2d1d5cb5e736d682e7716320ed37be034d4c855595777174e5ada9a: Status 404 returned error can't find the container with id c529bb60a2d1d5cb5e736d682e7716320ed37be034d4c855595777174e5ada9a Jul 08 19:03:20.442034 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.442000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:20.442208 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.442164 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 19:03:20.442251 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.442231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls podName:d08b88c4-dfe9-4a84-8911-86761acc6f9f nodeName:}" failed. No retries permitted until 2026-07-08 19:03:21.442214443 +0000 UTC m=+257.870102733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qk8d9" (UID: "d08b88c4-dfe9-4a84-8911-86761acc6f9f") : secret "samples-operator-tls" not found Jul 08 19:03:20.542314 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.542275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.542443 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.542410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:20.542443 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.542433 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:20.542513 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.542445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:20.542513 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.542493 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:21.542475221 +0000 UTC m=+257.970363530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:20.542582 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.542566 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:20.542636 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.542626 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:21.542609061 +0000 UTC m=+257.970497358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:20.542674 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.542663 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:21.542652493 +0000 UTC m=+257.970540778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:20.643041 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.642969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:20.643170 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.643125 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:20.643170 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.643141 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:20.643246 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:20.643184 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:21.643171714 +0000 UTC m=+258.071059999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:20.971560 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.971515 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" event={"ID":"0b75b4e4-3ce0-418d-90ba-fd705325ff1f","Type":"ContainerStarted","Data":"9264982c4dad0eb3849583154eaf8cd9236f44bd483d684efce913a6dbcc25d6"} Jul 08 19:03:20.973449 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.973408 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" event={"ID":"a86510c6-35a5-48f4-badb-aea0921624d9","Type":"ContainerStarted","Data":"f355b0e5d872cb9641c0bd0ad6b72347fd66021d95fb696242d10608c3031e69"} Jul 08 19:03:20.975272 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.975192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" event={"ID":"8d0494fa-e61e-45a7-b75b-6e9113bb91de","Type":"ContainerStarted","Data":"bd6a3440ae52d95d61f3015ba03b5d72b28053c2d7bb495396add08112384cae"} Jul 08 19:03:20.975272 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.975240 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" event={"ID":"8d0494fa-e61e-45a7-b75b-6e9113bb91de","Type":"ContainerStarted","Data":"c529bb60a2d1d5cb5e736d682e7716320ed37be034d4c855595777174e5ada9a"} Jul 08 19:03:20.993334 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:20.993179 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-t6rld" podStartSLOduration=1.99316037 podStartE2EDuration="1.99316037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:20.992357334 +0000 UTC m=+257.420245644" watchObservedRunningTime="2026-07-08 19:03:20.99316037 +0000 UTC m=+257.421048684" Jul 08 19:03:21.450597 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:21.450521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:21.450806 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.450698 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 19:03:21.450806 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.450774 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls podName:d08b88c4-dfe9-4a84-8911-86761acc6f9f nodeName:}" failed. No retries permitted until 2026-07-08 19:03:23.450753724 +0000 UTC m=+259.878642026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qk8d9" (UID: "d08b88c4-dfe9-4a84-8911-86761acc6f9f") : secret "samples-operator-tls" not found Jul 08 19:03:21.551225 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:21.551174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:21.551417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:21.551358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:21.551417 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.551393 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:23.551371591 +0000 UTC m=+259.979259880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:21.551556 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.551452 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:21.551556 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.551505 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:23.551486833 +0000 UTC m=+259.979375135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:21.551556 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:21.551541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:21.551712 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.551635 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:21.551712 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.551672 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:23.551660701 +0000 UTC m=+259.979548988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:21.652129 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:21.652074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:21.652371 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.652357 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:21.652440 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.652377 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:21.652496 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:21.652448 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:23.652426498 +0000 UTC m=+260.080314799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:22.983431 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:22.983391 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" event={"ID":"a86510c6-35a5-48f4-badb-aea0921624d9","Type":"ContainerStarted","Data":"90721a677e09e53444603df45f2db71ed16744f152282cf68ccefa6843fa544f"} Jul 08 19:03:23.005648 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.005598 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" podStartSLOduration=1.349385665 podStartE2EDuration="4.005583297s" totalImagesPullingTime="2.656197632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:23.003959524 +0000 UTC m=+259.431847856" watchObservedRunningTime="2026-07-08 19:03:23.005583297 +0000 UTC m=+259.433471604" Jul 08 19:03:23.469775 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.469736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:23.469955 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.469902 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 19:03:23.470026 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.469977 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls podName:d08b88c4-dfe9-4a84-8911-86761acc6f9f nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.469957323 +0000 UTC m=+263.897845614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qk8d9" (UID: "d08b88c4-dfe9-4a84-8911-86761acc6f9f") : secret "samples-operator-tls" not found Jul 08 19:03:23.570446 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.570410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:23.570605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.570502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:23.570605 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.570584 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.570560031 +0000 UTC m=+263.998448331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:23.570724 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.570640 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:23.570724 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.570655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:23.570724 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.570686 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.57067163 +0000 UTC m=+263.998559920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:23.570879 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.570751 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:23.570879 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.570797 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.570783023 +0000 UTC m=+263.998671308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:23.671587 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:23.671548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:23.671774 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.671679 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:23.671774 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.671692 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:23.671774 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:23.671745 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.671727313 +0000 UTC m=+264.099615620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:24.265770 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.265732 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hjtqk"] Jul 08 19:03:24.268024 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.268002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.274132 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.274108 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:03:24.275274 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.275162 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:03:24.277072 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.277056 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:03:24.282412 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.282398 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:03:24.287255 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.287238 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v5zgs\"" Jul 08 19:03:24.329810 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.329737 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjtqk"] Jul 08 19:03:24.376845 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.376821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.376970 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.376861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.376970 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.376906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vzvl\" (UniqueName: \"kubernetes.io/projected/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-api-access-7vzvl\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.377062 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.376989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-crio-socket\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.377062 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.377020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-data-volume\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477511 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477659 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477659 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477585 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vzvl\" (UniqueName: \"kubernetes.io/projected/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-api-access-7vzvl\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477659 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-crio-socket\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477659 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.477632 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:24.477659 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-data-volume\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477878 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.477682 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:03:24.477878 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.477696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:24.977678577 +0000 UTC m=+261.405566930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:24.477878 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-crio-socket\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.477878 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.477747 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:24.977728204 +0000 UTC m=+261.405616491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : configmap "kube-rbac-proxy" not found Jul 08 19:03:24.478007 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.477987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-data-volume\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.487114 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.487073 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vzvl\" (UniqueName: \"kubernetes.io/projected/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-api-access-7vzvl\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.982202 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.982158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.982397 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.982241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:24.982397 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.982317 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:24.982397 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.982351 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:03:24.982397 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.982387 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:25.982371737 +0000 UTC m=+262.410260023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:24.982628 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:24.982402 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:25.982395853 +0000 UTC m=+262.410284138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : configmap "kube-rbac-proxy" not found Jul 08 19:03:24.991112 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:24.991077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" event={"ID":"0b75b4e4-3ce0-418d-90ba-fd705325ff1f","Type":"ContainerStarted","Data":"7aff02ec7ab045e9e789dd7ca727f6a65ded8986ce4c0e10abfebea00487b721"} Jul 08 19:03:25.009781 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:25.009739 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" podStartSLOduration=2.138000066 podStartE2EDuration="6.009726569s" totalImagesPullingTime="3.871726503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:25.009025726 +0000 UTC m=+261.436914038" watchObservedRunningTime="2026-07-08 19:03:25.009726569 +0000 UTC m=+261.437614877" Jul 08 19:03:25.992400 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:25.992283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:25.992400 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:25.992364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:25.992838 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:25.992449 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:25.992838 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:25.992449 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:03:25.992838 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:25.992525 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.992509025 +0000 UTC m=+264.420397310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:25.992838 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:25.992616 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:27.992594297 +0000 UTC m=+264.420482596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : configmap "kube-rbac-proxy" not found Jul 08 19:03:26.189418 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:26.189388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rtnkj_270c8cba-3f78-4663-85e1-ba8d92d58158/dns-node-resolver/0.log" Jul 08 19:03:27.189108 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.189082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-655q2_58595100-3eee-4a74-9476-e81e8609049c/node-ca/0.log" Jul 08 19:03:27.505650 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.505617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:27.505818 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.505769 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 08 19:03:27.505856 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.505829 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls podName:d08b88c4-dfe9-4a84-8911-86761acc6f9f nodeName:}" failed. No retries permitted until 2026-07-08 19:03:35.505813843 +0000 UTC m=+271.933702129 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-qk8d9" (UID: "d08b88c4-dfe9-4a84-8911-86761acc6f9f") : secret "samples-operator-tls" not found Jul 08 19:03:27.606139 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.606109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:27.606289 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.606155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:27.606289 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.606238 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:27.606289 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.606251 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:35.606235278 +0000 UTC m=+272.034123564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:27.606289 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.606288 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:35.606271033 +0000 UTC m=+272.034159319 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:27.606452 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.606378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:27.606531 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.606515 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:27.606565 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.606559 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:35.606549149 +0000 UTC m=+272.034437439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:27.707223 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.707191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:27.707376 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.707360 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:27.707429 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.707378 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:27.707467 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:27.707431 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:35.707416411 +0000 UTC m=+272.135304697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:27.989879 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:27.989851 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n9mqf_b4a10718-e056-4a7a-851b-be327aff9606/serve-healthcheck-canary/0.log" Jul 08 19:03:28.009923 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:28.009891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:28.010092 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:28.009949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:28.010092 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:28.010019 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:03:28.010092 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:28.010085 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:32.010066249 +0000 UTC m=+268.437954537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : configmap "kube-rbac-proxy" not found Jul 08 19:03:28.010257 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:28.010022 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:28.010257 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:28.010150 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:32.010137149 +0000 UTC m=+268.438025435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:32.044358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:32.044292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:32.044358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:32.044373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:32.044831 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:32.044444 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:32.044831 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:32.044491 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:03:32.044831 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:32.044510 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:40.044494074 +0000 UTC m=+276.472382364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:32.044831 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:32.044537 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:40.044525156 +0000 UTC m=+276.472413442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : configmap "kube-rbac-proxy" not found Jul 08 19:03:35.573757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.573716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:35.576095 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.576072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d08b88c4-dfe9-4a84-8911-86761acc6f9f-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-qk8d9\" (UID: \"d08b88c4-dfe9-4a84-8911-86761acc6f9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:35.656681 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.656652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" Jul 08 19:03:35.674252 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.674226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:35.674363 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.674341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:35.674414 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.674365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:35.674414 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.674392 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 19:03:35.674492 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.674456 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:51.674436554 +0000 UTC m=+288.102324845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : secret "router-metrics-certs-default" not found Jul 08 19:03:35.674492 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.674484 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:35.674572 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.674517 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle podName:125a062b-4f51-4040-bb38-f0a8a610d369 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:51.674500141 +0000 UTC m=+288.102388446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle") pod "router-default-7cf956c5dc-r496w" (UID: "125a062b-4f51-4040-bb38-f0a8a610d369") : configmap references non-existent config key: service-ca.crt Jul 08 19:03:35.674572 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.674535 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls podName:a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b nodeName:}" failed. No retries permitted until 2026-07-08 19:03:51.674526158 +0000 UTC m=+288.102414447 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-fjb2n" (UID: "a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b") : secret "cluster-monitoring-operator-tls" not found Jul 08 19:03:35.774986 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.774951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:35.775133 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.775085 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:03:35.775133 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.775104 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dcdfc7-gnpfk: secret "image-registry-tls" not found Jul 08 19:03:35.775217 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:35.775164 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls podName:af265bef-5604-491a-a170-6a9ef21c667c nodeName:}" failed. No retries permitted until 2026-07-08 19:03:51.775141734 +0000 UTC m=+288.203030023 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls") pod "image-registry-6dcdfc7-gnpfk" (UID: "af265bef-5604-491a-a170-6a9ef21c667c") : secret "image-registry-tls" not found Jul 08 19:03:35.780898 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:35.780876 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9"] Jul 08 19:03:36.021825 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:36.021795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" event={"ID":"d08b88c4-dfe9-4a84-8911-86761acc6f9f","Type":"ContainerStarted","Data":"dac136d003faf4e46bed6cfe1cd8431393259044fa25ef1e3ee5a8d234c7307e"} Jul 08 19:03:39.031215 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:39.031174 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" event={"ID":"d08b88c4-dfe9-4a84-8911-86761acc6f9f","Type":"ContainerStarted","Data":"c1ce88e9a6295af72aedc0a4ddfd84ed40f7a404c690c730516f50eaa39d9cef"} Jul 08 19:03:39.031699 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:39.031290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" event={"ID":"d08b88c4-dfe9-4a84-8911-86761acc6f9f","Type":"ContainerStarted","Data":"8e4e3e6354316724e832416f1fed6f634aabb78191a0e079b7501c233a09d95c"} Jul 08 19:03:39.051972 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:39.051917 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-qk8d9" podStartSLOduration=17.595741019 podStartE2EDuration="20.051901199s" totalImagesPullingTime="2.45616018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:39.051508874 +0000 UTC m=+275.479397184" watchObservedRunningTime="2026-07-08 19:03:39.051901199 +0000 UTC m=+275.479789509" Jul 08 19:03:40.107868 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:40.107830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:40.108345 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:40.107890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:40.108345 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:40.107974 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:03:40.108345 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:40.108080 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls podName:c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea nodeName:}" failed. No retries permitted until 2026-07-08 19:03:56.108058425 +0000 UTC m=+292.535946721 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjtqk" (UID: "c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea") : secret "insights-runtime-extractor-tls" not found Jul 08 19:03:40.108473 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:40.108456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:44.702047 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:44.702006 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vdx2v" podUID="a5a50da0-90dd-407a-a1bf-b57bd756124e" Jul 08 19:03:45.048722 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:45.048694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdx2v" Jul 08 19:03:48.265797 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:48.265763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 19:03:48.268020 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:48.268000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5a50da0-90dd-407a-a1bf-b57bd756124e-metrics-tls\") pod \"dns-default-vdx2v\" (UID: \"a5a50da0-90dd-407a-a1bf-b57bd756124e\") " pod="openshift-dns/dns-default-vdx2v" Jul 08 19:03:48.353152 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:48.353120 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-97fdz\"" Jul 08 19:03:48.360714 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:48.360690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vdx2v" Jul 08 19:03:48.493931 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:48.493899 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vdx2v"] Jul 08 19:03:48.497046 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:48.497015 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a50da0_90dd_407a_a1bf_b57bd756124e.slice/crio-4449d4d5aed406629a63107a86266d066ff8024d9992c2a3cde1e279395e3b48 WatchSource:0}: Error finding container 4449d4d5aed406629a63107a86266d066ff8024d9992c2a3cde1e279395e3b48: Status 404 returned error can't find the container with id 4449d4d5aed406629a63107a86266d066ff8024d9992c2a3cde1e279395e3b48 Jul 08 19:03:49.060822 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:49.060785 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdx2v" event={"ID":"a5a50da0-90dd-407a-a1bf-b57bd756124e","Type":"ContainerStarted","Data":"4449d4d5aed406629a63107a86266d066ff8024d9992c2a3cde1e279395e3b48"} Jul 08 19:03:51.068930 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.068898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdx2v" event={"ID":"a5a50da0-90dd-407a-a1bf-b57bd756124e","Type":"ContainerStarted","Data":"9e40108613affddcb6345aad8710acd2b037dd04257594c897a2d1c9a8b728e7"} Jul 08 19:03:51.069317 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.068954 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vdx2v" event={"ID":"a5a50da0-90dd-407a-a1bf-b57bd756124e","Type":"ContainerStarted","Data":"9d8adffef946afc5f3b1e19e285fe7b6fa87c17b5d8c5df3e34035c2780667bc"} Jul 08 19:03:51.069317 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.069027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vdx2v" Jul 08 19:03:51.095598 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.095536 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vdx2v" podStartSLOduration=251.447416989 podStartE2EDuration="4m13.095517534s" totalImagesPullingTime="1.648100545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:51.093857966 +0000 UTC m=+287.521746275" watchObservedRunningTime="2026-07-08 19:03:51.095517534 +0000 UTC m=+287.523405842" Jul 08 19:03:51.694828 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.694794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:51.695005 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.694857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:51.695005 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.694884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:51.695576 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.695551 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/125a062b-4f51-4040-bb38-f0a8a610d369-service-ca-bundle\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:51.697208 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.697179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-fjb2n\" (UID: \"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:51.697324 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.697235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125a062b-4f51-4040-bb38-f0a8a610d369-metrics-certs\") pod \"router-default-7cf956c5dc-r496w\" (UID: \"125a062b-4f51-4040-bb38-f0a8a610d369\") " pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:51.795757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.795726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:51.797970 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.797952 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"image-registry-6dcdfc7-gnpfk\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:51.866472 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.866441 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:03:51.866734 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.866719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:51.930759 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.930677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:51.937956 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:51.937924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" Jul 08 19:03:52.017809 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:52.017784 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:03:52.019703 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:52.019670 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf265bef_5604_491a_a170_6a9ef21c667c.slice/crio-250ab99f043c19c0a950e3b6f912bb593b26300640c76b99b34a4b63895a55c2 WatchSource:0}: Error finding container 250ab99f043c19c0a950e3b6f912bb593b26300640c76b99b34a4b63895a55c2: Status 404 returned error can't find the container with id 250ab99f043c19c0a950e3b6f912bb593b26300640c76b99b34a4b63895a55c2 Jul 08 19:03:52.072926 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:52.072897 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" event={"ID":"af265bef-5604-491a-a170-6a9ef21c667c","Type":"ContainerStarted","Data":"250ab99f043c19c0a950e3b6f912bb593b26300640c76b99b34a4b63895a55c2"} Jul 08 19:03:52.083982 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:52.083884 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cf956c5dc-r496w"] Jul 08 19:03:52.086176 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:52.086150 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125a062b_4f51_4040_bb38_f0a8a610d369.slice/crio-f73853f999949a3b18b81beed0f7c0d908dee6ee274cdf978db3f34e3253f7aa WatchSource:0}: Error finding container f73853f999949a3b18b81beed0f7c0d908dee6ee274cdf978db3f34e3253f7aa: Status 404 returned error can't find the container with id f73853f999949a3b18b81beed0f7c0d908dee6ee274cdf978db3f34e3253f7aa Jul 08 19:03:52.098144 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:52.098122 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n"] Jul 08 19:03:52.101046 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:52.101024 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda424b5c6_1fc5_44fd_bc0f_7572c9db3b7b.slice/crio-275eeee4e6e4743850d94b7da11e82176d355304603209ad6a742804e1fce0fe WatchSource:0}: Error finding container 275eeee4e6e4743850d94b7da11e82176d355304603209ad6a742804e1fce0fe: Status 404 returned error can't find the container with id 275eeee4e6e4743850d94b7da11e82176d355304603209ad6a742804e1fce0fe Jul 08 19:03:53.077925 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.077889 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" event={"ID":"af265bef-5604-491a-a170-6a9ef21c667c","Type":"ContainerStarted","Data":"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4"} Jul 08 19:03:53.078406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.077949 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:03:53.079048 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.079023 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" event={"ID":"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b","Type":"ContainerStarted","Data":"275eeee4e6e4743850d94b7da11e82176d355304603209ad6a742804e1fce0fe"} Jul 08 19:03:53.080631 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.080604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cf956c5dc-r496w" event={"ID":"125a062b-4f51-4040-bb38-f0a8a610d369","Type":"ContainerStarted","Data":"8b356502f4c96613d1c078abdcdbc16d60c1e598c8a69c6abcf8446672771fb9"} Jul 08 19:03:53.080747 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.080654 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cf956c5dc-r496w" event={"ID":"125a062b-4f51-4040-bb38-f0a8a610d369","Type":"ContainerStarted","Data":"f73853f999949a3b18b81beed0f7c0d908dee6ee274cdf978db3f34e3253f7aa"} Jul 08 19:03:53.107145 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.107105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" podStartSLOduration=34.107092788 podStartE2EDuration="34.107092788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:53.105642516 +0000 UTC m=+289.533530823" watchObservedRunningTime="2026-07-08 19:03:53.107092788 +0000 UTC m=+289.534981075" Jul 08 19:03:53.143972 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.143934 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7cf956c5dc-r496w" podStartSLOduration=34.143919903 podStartE2EDuration="34.143919903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:53.142762725 +0000 UTC m=+289.570651033" watchObservedRunningTime="2026-07-08 19:03:53.143919903 +0000 UTC m=+289.571808247" Jul 08 19:03:53.931601 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.931566 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:53.934382 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:53.934357 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:54.085663 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:54.085621 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:54.087058 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:54.087036 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cf956c5dc-r496w" Jul 08 19:03:55.089951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:55.089905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" event={"ID":"a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b","Type":"ContainerStarted","Data":"cb3c9dab56a845b1d408c1b5f4309d09701aef6f94353d9d0f89a4a202a0e422"} Jul 08 19:03:55.107188 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:55.107105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-fjb2n" podStartSLOduration=33.983409797 podStartE2EDuration="36.107087775s" totalImagesPullingTime="2.123677978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:03:55.107087103 +0000 UTC m=+291.534975411" watchObservedRunningTime="2026-07-08 19:03:55.107087775 +0000 UTC m=+291.534976088" Jul 08 19:03:56.133634 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:56.133596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:56.135948 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:56.135921 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjtqk\" (UID: \"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea\") " pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:56.377907 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:56.377862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjtqk" Jul 08 19:03:56.518061 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:56.518032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjtqk"] Jul 08 19:03:56.521137 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:03:56.521108 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ea5ba4_7dae_477a_abfe_5e1e875cc4ea.slice/crio-70b89c9247ea2e0cb9f44bac66ebabcd872d4dc1fba490b35e2e552772bb7624 WatchSource:0}: Error finding container 70b89c9247ea2e0cb9f44bac66ebabcd872d4dc1fba490b35e2e552772bb7624: Status 404 returned error can't find the container with id 70b89c9247ea2e0cb9f44bac66ebabcd872d4dc1fba490b35e2e552772bb7624 Jul 08 19:03:57.099341 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.099283 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjtqk" event={"ID":"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea","Type":"ContainerStarted","Data":"70b89c9247ea2e0cb9f44bac66ebabcd872d4dc1fba490b35e2e552772bb7624"} Jul 08 19:03:57.210347 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:57.210279 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9gmnq" podUID="7578c673-39a4-4942-9470-e85a33f20c0e" Jul 08 19:03:57.799548 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.799512 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz"] Jul 08 19:03:57.802463 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.802441 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.806586 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.806560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4bhw6\"" Jul 08 19:03:57.806683 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.806658 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 19:03:57.806804 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.806789 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 19:03:57.806844 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.806567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:03:57.817101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.817076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz"] Jul 08 19:03:57.848763 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.848736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.848884 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.848776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.848884 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.848804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vzzk\" (UniqueName: \"kubernetes.io/projected/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-kube-api-access-8vzzk\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.848981 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.848886 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.949506 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.949477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.949633 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.949579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.949633 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.949609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.949830 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.949635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vzzk\" (UniqueName: \"kubernetes.io/projected/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-kube-api-access-8vzzk\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.949830 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:57.949761 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 19:03:57.949994 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:03:57.949845 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls podName:38aba6cc-6f1b-40e9-bab3-81c0c95babe2 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:58.449824957 +0000 UTC m=+294.877713257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-q4sdz" (UID: "38aba6cc-6f1b-40e9-bab3-81c0c95babe2") : secret "prometheus-operator-tls" not found Jul 08 19:03:57.950438 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.950419 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.952143 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.952123 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:57.959115 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:57.959080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vzzk\" (UniqueName: \"kubernetes.io/projected/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-kube-api-access-8vzzk\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:58.453006 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:58.452967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:58.455645 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:58.455622 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38aba6cc-6f1b-40e9-bab3-81c0c95babe2-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-q4sdz\" (UID: \"38aba6cc-6f1b-40e9-bab3-81c0c95babe2\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:03:58.713773 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:03:58.713739 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" Jul 08 19:04:00.308966 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:00.308935 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz"] Jul 08 19:04:00.312734 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:04:00.312704 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38aba6cc_6f1b_40e9_bab3_81c0c95babe2.slice/crio-c17250c6df72c3aef50403b14bfc0295b4062a8c21fba9f54b93d135bb0fae41 WatchSource:0}: Error finding container c17250c6df72c3aef50403b14bfc0295b4062a8c21fba9f54b93d135bb0fae41: Status 404 returned error can't find the container with id c17250c6df72c3aef50403b14bfc0295b4062a8c21fba9f54b93d135bb0fae41 Jul 08 19:04:01.075475 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:01.075454 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vdx2v" Jul 08 19:04:01.116069 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:01.116033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjtqk" event={"ID":"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea","Type":"ContainerStarted","Data":"8071818e17ff792d9cb9f9a864ae221eb5ad691f08241b1a33bdf15c241960ee"} Jul 08 19:04:01.116201 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:01.116135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjtqk" event={"ID":"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea","Type":"ContainerStarted","Data":"e9bd8bb8eb034f4352e0e8b9fbdbf0f743955745700175525fc12670b67041f8"} Jul 08 19:04:01.117070 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:01.117048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" event={"ID":"38aba6cc-6f1b-40e9-bab3-81c0c95babe2","Type":"ContainerStarted","Data":"c17250c6df72c3aef50403b14bfc0295b4062a8c21fba9f54b93d135bb0fae41"} Jul 08 19:04:04.014908 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.014884 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:04:04.014908 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.014895 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:04:04.019330 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.019295 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:04:04.129800 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.129770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjtqk" event={"ID":"c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea","Type":"ContainerStarted","Data":"9650a8f104788be50b544bdd8bd8483aee9027ddc9b051e80647677d463bc4d6"} Jul 08 19:04:04.131353 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.131324 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" event={"ID":"38aba6cc-6f1b-40e9-bab3-81c0c95babe2","Type":"ContainerStarted","Data":"44466990a8b82eae2d492ffb863e4895ae7b7339c2c27d6cde5d9353ea9828af"} Jul 08 19:04:04.131477 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.131370 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" event={"ID":"38aba6cc-6f1b-40e9-bab3-81c0c95babe2","Type":"ContainerStarted","Data":"4d1f3e86060e2db3d1f05d8b4998cbc67ad755336410a06cf1f2de5f9750b580"} Jul 08 19:04:04.152565 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.152524 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hjtqk" podStartSLOduration=33.10065144 podStartE2EDuration="40.152510201s" totalImagesPullingTime="7.051858761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:04.151896435 +0000 UTC m=+300.579784743" watchObservedRunningTime="2026-07-08 19:04:04.152510201 +0000 UTC m=+300.580398509" Jul 08 19:04:04.174095 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:04.174058 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-q4sdz" podStartSLOduration=3.800853586 podStartE2EDuration="7.174021444s" totalImagesPullingTime="3.373167858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:04.1730695 +0000 UTC m=+300.600957810" watchObservedRunningTime="2026-07-08 19:04:04.174021444 +0000 UTC m=+300.601909752" Jul 08 19:04:06.251030 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.250989 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7t66q"] Jul 08 19:04:06.253836 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.253814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.256764 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.256745 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s95pz\"" Jul 08 19:04:06.257738 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.257721 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:04:06.258157 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.258139 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:04:06.258414 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.258395 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:04:06.320807 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-wtmp\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.320939 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-sys\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.320939 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-root\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.320939 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.320939 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.321086 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-metrics-client-ca\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.321086 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.320985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.321086 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.321026 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-textfile\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.321086 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.321043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hpjz\" (UniqueName: \"kubernetes.io/projected/e3a25cd9-e504-4330-9245-a9c0d1f4218b-kube-api-access-8hpjz\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.421859 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.421832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-metrics-client-ca\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422032 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.421865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422032 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.421893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-textfile\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422032 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.421908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hpjz\" (UniqueName: \"kubernetes.io/projected/e3a25cd9-e504-4330-9245-a9c0d1f4218b-kube-api-access-8hpjz\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422032 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.421939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-wtmp\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422032 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:04:06.421993 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422060 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-wtmp\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:04:06.422084 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls podName:e3a25cd9-e504-4330-9245-a9c0d1f4218b nodeName:}" failed. No retries permitted until 2026-07-08 19:04:06.922063136 +0000 UTC m=+303.349951423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls") pod "node-exporter-7t66q" (UID: "e3a25cd9-e504-4330-9245-a9c0d1f4218b") : secret "node-exporter-tls" not found Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-sys\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-root\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-sys\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-textfile\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422398 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3a25cd9-e504-4330-9245-a9c0d1f4218b-root\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422481 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-metrics-client-ca\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.422737 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.422720 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-accelerators-collector-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.424362 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.424342 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.432806 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.432785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hpjz\" (UniqueName: \"kubernetes.io/projected/e3a25cd9-e504-4330-9245-a9c0d1f4218b-kube-api-access-8hpjz\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.926950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.926912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:06.929348 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:06.929325 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3a25cd9-e504-4330-9245-a9c0d1f4218b-node-exporter-tls\") pod \"node-exporter-7t66q\" (UID: \"e3a25cd9-e504-4330-9245-a9c0d1f4218b\") " pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:07.163031 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:07.162991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7t66q" Jul 08 19:04:07.173235 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:04:07.173207 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a25cd9_e504_4330_9245_a9c0d1f4218b.slice/crio-75ffd79266396c9cca3ffb9527bb94c196c529dc294dc6c9ebd10d27b132c517 WatchSource:0}: Error finding container 75ffd79266396c9cca3ffb9527bb94c196c529dc294dc6c9ebd10d27b132c517: Status 404 returned error can't find the container with id 75ffd79266396c9cca3ffb9527bb94c196c529dc294dc6c9ebd10d27b132c517 Jul 08 19:04:07.175171 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:07.175150 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:04:08.145334 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:08.145286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7t66q" event={"ID":"e3a25cd9-e504-4330-9245-a9c0d1f4218b","Type":"ContainerStarted","Data":"75ffd79266396c9cca3ffb9527bb94c196c529dc294dc6c9ebd10d27b132c517"} Jul 08 19:04:09.150722 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.150686 2580 generic.go:356] "Generic (PLEG): container finished" podID="e3a25cd9-e504-4330-9245-a9c0d1f4218b" containerID="2140fea351cd49358b49b469bf33649537e34746857d76576d7ff29bbc14132b" exitCode=0 Jul 08 19:04:09.151141 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.150742 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7t66q" event={"ID":"e3a25cd9-e504-4330-9245-a9c0d1f4218b","Type":"ContainerDied","Data":"2140fea351cd49358b49b469bf33649537e34746857d76576d7ff29bbc14132b"} Jul 08 19:04:09.200006 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.199981 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f4948666f-75pkn"] Jul 08 19:04:09.203420 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.203400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.206149 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.206128 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 19:04:09.206149 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.206138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 19:04:09.206149 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.206146 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-14qnhpdj1vc93\"" Jul 08 19:04:09.206503 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.206484 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 19:04:09.206749 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.206735 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-wnvrq\"" Jul 08 19:04:09.207073 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.207058 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 19:04:09.207257 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.207243 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 19:04:09.230962 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.230935 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f4948666f-75pkn"] Jul 08 19:04:09.348133 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-grpc-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348281 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348281 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348281 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf71693-6f61-4e64-a6d4-7296c19b656f-metrics-client-ca\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348281 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348499 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348499 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87vrw\" (UniqueName: \"kubernetes.io/projected/2bf71693-6f61-4e64-a6d4-7296c19b656f-kube-api-access-87vrw\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.348499 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.348415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-grpc-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449851 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf71693-6f61-4e64-a6d4-7296c19b656f-metrics-client-ca\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449851 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449632 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449851 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.449851 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87vrw\" (UniqueName: \"kubernetes.io/projected/2bf71693-6f61-4e64-a6d4-7296c19b656f-kube-api-access-87vrw\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.450058 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.449849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.450734 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.450691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bf71693-6f61-4e64-a6d4-7296c19b656f-metrics-client-ca\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452355 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452293 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452453 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452569 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-grpc-tls\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452652 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452713 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.452764 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.452733 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bf71693-6f61-4e64-a6d4-7296c19b656f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.464001 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.463978 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87vrw\" (UniqueName: \"kubernetes.io/projected/2bf71693-6f61-4e64-a6d4-7296c19b656f-kube-api-access-87vrw\") pod \"thanos-querier-5f4948666f-75pkn\" (UID: \"2bf71693-6f61-4e64-a6d4-7296c19b656f\") " pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.511993 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.511973 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:09.642569 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:09.642540 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f4948666f-75pkn"] Jul 08 19:04:09.645458 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:04:09.645428 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf71693_6f61_4e64_a6d4_7296c19b656f.slice/crio-7b0618c8d0dc3297563573a45d8a94de864269f5e7f6db215c77e6d6450e61a3 WatchSource:0}: Error finding container 7b0618c8d0dc3297563573a45d8a94de864269f5e7f6db215c77e6d6450e61a3: Status 404 returned error can't find the container with id 7b0618c8d0dc3297563573a45d8a94de864269f5e7f6db215c77e6d6450e61a3 Jul 08 19:04:10.157395 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.157357 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7t66q" event={"ID":"e3a25cd9-e504-4330-9245-a9c0d1f4218b","Type":"ContainerStarted","Data":"8ffce7261a63a48d972e0212d36cc1ad8c16f987cae8e6017e20fd63fbb077a7"} Jul 08 19:04:10.157801 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.157421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7t66q" event={"ID":"e3a25cd9-e504-4330-9245-a9c0d1f4218b","Type":"ContainerStarted","Data":"fe7d33122808b512abd935970a4dfa2c7104c0653ff03a59f2aa38581a34543c"} Jul 08 19:04:10.158494 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.158475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"7b0618c8d0dc3297563573a45d8a94de864269f5e7f6db215c77e6d6450e61a3"} Jul 08 19:04:10.184163 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.184115 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7t66q" podStartSLOduration=2.979995815 podStartE2EDuration="4.184101331s" totalImagesPullingTime="1.137893952s" totalInitContainerRuntime="66.211564ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:10.182944882 +0000 UTC m=+306.610833187" watchObservedRunningTime="2026-07-08 19:04:10.184101331 +0000 UTC m=+306.611989640" Jul 08 19:04:10.957452 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.957416 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn"] Jul 08 19:04:10.959984 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.959964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:10.962756 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.962733 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lg978\"" Jul 08 19:04:10.962880 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.962733 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 19:04:10.969433 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:10.969380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn"] Jul 08 19:04:11.062716 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.062687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.062855 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.062724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flmpv\" (UniqueName: \"kubernetes.io/projected/7c044123-79f8-412e-93a5-26afda868b32-kube-api-access-flmpv\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.163445 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.163413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.163864 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.163464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flmpv\" (UniqueName: \"kubernetes.io/projected/7c044123-79f8-412e-93a5-26afda868b32-kube-api-access-flmpv\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.163864 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:04:11.163574 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 19:04:11.163864 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:04:11.163641 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert podName:7c044123-79f8-412e-93a5-26afda868b32 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:11.663622194 +0000 UTC m=+308.091510497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-gm4vn" (UID: "7c044123-79f8-412e-93a5-26afda868b32") : secret "monitoring-plugin-cert" not found Jul 08 19:04:11.173641 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.173611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flmpv\" (UniqueName: \"kubernetes.io/projected/7c044123-79f8-412e-93a5-26afda868b32-kube-api-access-flmpv\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.208532 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.208463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:04:11.669416 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.669384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.672102 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.672076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c044123-79f8-412e-93a5-26afda868b32-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gm4vn\" (UID: \"7c044123-79f8-412e-93a5-26afda868b32\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:11.871681 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:11.871649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:12.405593 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:12.405564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn"] Jul 08 19:04:12.408605 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:04:12.408562 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c044123_79f8_412e_93a5_26afda868b32.slice/crio-1de758c5a586487885434382c7bcd188815dbdf569a629b2c692ed20d829dcc6 WatchSource:0}: Error finding container 1de758c5a586487885434382c7bcd188815dbdf569a629b2c692ed20d829dcc6: Status 404 returned error can't find the container with id 1de758c5a586487885434382c7bcd188815dbdf569a629b2c692ed20d829dcc6 Jul 08 19:04:13.085009 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:13.084982 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:04:13.175069 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:13.175030 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" event={"ID":"7c044123-79f8-412e-93a5-26afda868b32","Type":"ContainerStarted","Data":"1de758c5a586487885434382c7bcd188815dbdf569a629b2c692ed20d829dcc6"} Jul 08 19:04:13.178431 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:13.178401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"f4a2319b0de04a8466c1dda2de950c29467b4ceffb10929c8c64b8d329b01660"} Jul 08 19:04:13.178574 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:13.178463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"35bbb72fc88f275d6238a393a1b83bf2401a979e8b4a6f6aaf7a5586ce3ab0c7"} Jul 08 19:04:13.178574 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:13.178490 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"3f614bf4443be80b78bc94a62c713e50c6bbab33822c4e1945ed91892e04d5dc"} Jul 08 19:04:14.183095 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.183011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" event={"ID":"7c044123-79f8-412e-93a5-26afda868b32","Type":"ContainerStarted","Data":"b991363230205cb587a8348d3b5e944693a71ebbe62ee04cace8c1aff6d6938f"} Jul 08 19:04:14.183523 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.183220 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:14.186180 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.186158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"496a688f00ba393da1027944654488f72718d86accd2fd596fab63d0d624d8e7"} Jul 08 19:04:14.186321 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.186210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"38ea8ec690d7e18313fe5f8d1c1e599df1ba8d9cee97994264dce131f4d39364"} Jul 08 19:04:14.186321 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.186236 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" event={"ID":"2bf71693-6f61-4e64-a6d4-7296c19b656f","Type":"ContainerStarted","Data":"1d3545397d70c1015701e74a40c586f82f0c3038730a6a0e3da22dd2d0228a55"} Jul 08 19:04:14.186424 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.186351 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:14.188378 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.188360 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" Jul 08 19:04:14.204056 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.204017 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gm4vn" podStartSLOduration=2.776275726 podStartE2EDuration="4.204004369s" totalImagesPullingTime="1.427728643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:14.203335054 +0000 UTC m=+310.631223358" watchObservedRunningTime="2026-07-08 19:04:14.204004369 +0000 UTC m=+310.631892655" Jul 08 19:04:14.265238 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:14.265194 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" podStartSLOduration=1.301580899 podStartE2EDuration="5.265181198s" totalImagesPullingTime="3.963600299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:14.264829036 +0000 UTC m=+310.692717345" watchObservedRunningTime="2026-07-08 19:04:14.265181198 +0000 UTC m=+310.693069549" Jul 08 19:04:18.100589 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.100544 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" podUID="af265bef-5604-491a-a170-6a9ef21c667c" containerName="registry" containerID="cri-o://c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4" gracePeriod=30 Jul 08 19:04:18.336071 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.336048 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:04:18.426180 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426110 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426180 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426148 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426180 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426204 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426231 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426263 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgnb6\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426323 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426367 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets\") pod \"af265bef-5604-491a-a170-6a9ef21c667c\" (UID: \"af265bef-5604-491a-a170-6a9ef21c667c\") " Jul 08 19:04:18.426708 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426518 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:04:18.426708 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426612 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-registry-certificates\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.427001 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.426966 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:04:18.428789 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.428734 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:04:18.428899 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.428847 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:04:18.428899 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.428856 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6" (OuterVolumeSpecName: "kube-api-access-qgnb6") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "kube-api-access-qgnb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:04:18.428899 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.428888 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:04:18.429061 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.428930 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:04:18.434882 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.434854 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "af265bef-5604-491a-a170-6a9ef21c667c" (UID: "af265bef-5604-491a-a170-6a9ef21c667c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:04:18.527583 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527557 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-image-registry-private-configuration\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527591 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af265bef-5604-491a-a170-6a9ef21c667c-trusted-ca\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527606 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/af265bef-5604-491a-a170-6a9ef21c667c-ca-trust-extracted\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527615 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgnb6\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-kube-api-access-qgnb6\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527624 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-registry-tls\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527632 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/af265bef-5604-491a-a170-6a9ef21c667c-installation-pull-secrets\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:18.527671 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:18.527640 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af265bef-5604-491a-a170-6a9ef21c667c-bound-sa-token\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:04:19.205579 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.205546 2580 generic.go:356] "Generic (PLEG): container finished" podID="af265bef-5604-491a-a170-6a9ef21c667c" containerID="c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4" exitCode=0 Jul 08 19:04:19.205968 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.205594 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" Jul 08 19:04:19.205968 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.205617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" event={"ID":"af265bef-5604-491a-a170-6a9ef21c667c","Type":"ContainerDied","Data":"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4"} Jul 08 19:04:19.205968 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.205654 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dcdfc7-gnpfk" event={"ID":"af265bef-5604-491a-a170-6a9ef21c667c","Type":"ContainerDied","Data":"250ab99f043c19c0a950e3b6f912bb593b26300640c76b99b34a4b63895a55c2"} Jul 08 19:04:19.205968 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.205674 2580 scope.go:117] "RemoveContainer" containerID="c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4" Jul 08 19:04:19.215377 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.215360 2580 scope.go:117] "RemoveContainer" containerID="c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4" Jul 08 19:04:19.215652 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:04:19.215632 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4\": container with ID starting with c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4 not found: ID does not exist" containerID="c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4" Jul 08 19:04:19.215700 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.215663 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4"} err="failed to get container status \"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4\": rpc error: code = NotFound desc = could not find container \"c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4\": container with ID starting with c00c3b64a4188ac24da57c783662829cb7da9ad6b0cc765ee3e0e168c54595a4 not found: ID does not exist" Jul 08 19:04:19.227568 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.227546 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:04:19.231789 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:19.231771 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dcdfc7-gnpfk"] Jul 08 19:04:20.196498 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:20.196468 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f4948666f-75pkn" Jul 08 19:04:20.212403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:20.212376 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af265bef-5604-491a-a170-6a9ef21c667c" path="/var/lib/kubelet/pods/af265bef-5604-491a-a170-6a9ef21c667c/volumes" Jul 08 19:04:24.803890 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.803861 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:04:24.804262 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.804137 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af265bef-5604-491a-a170-6a9ef21c667c" containerName="registry" Jul 08 19:04:24.804262 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.804148 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="af265bef-5604-491a-a170-6a9ef21c667c" containerName="registry" Jul 08 19:04:24.804262 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.804207 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="af265bef-5604-491a-a170-6a9ef21c667c" containerName="registry" Jul 08 19:04:24.808178 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.808156 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.811825 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.811801 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:04:24.812951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.812930 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:04:24.812951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.812945 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:04:24.813156 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.813140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 19:04:24.813323 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.813288 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 19:04:24.813431 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.813325 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:04:24.813495 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.813428 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:04:24.814191 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.814175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-659rr\"" Jul 08 19:04:24.821378 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.821355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:04:24.821661 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.821642 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:04:24.881102 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881207 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881207 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881207 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881334 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881334 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.881402 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.881348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jztz\" (UniqueName: \"kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981800 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981800 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981869 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jztz\" (UniqueName: \"kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.981950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.981940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.982801 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.982778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.982900 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.982878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.982946 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.982893 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.983105 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.983088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.984322 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.984283 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.984470 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.984451 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:24.990315 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:24.990280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jztz\" (UniqueName: \"kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz\") pod \"console-845c9d497b-sfzd8\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:25.117620 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:25.117550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:25.249040 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:25.249012 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:04:25.251467 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:04:25.251439 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17653de6_6dde_4568_9b13_6504d200be01.slice/crio-1572dd3a92f3f31add84734c72f62a2a3ca140738c623a94651a16d3989f95e2 WatchSource:0}: Error finding container 1572dd3a92f3f31add84734c72f62a2a3ca140738c623a94651a16d3989f95e2: Status 404 returned error can't find the container with id 1572dd3a92f3f31add84734c72f62a2a3ca140738c623a94651a16d3989f95e2 Jul 08 19:04:26.232085 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:26.232034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845c9d497b-sfzd8" event={"ID":"17653de6-6dde-4568-9b13-6504d200be01","Type":"ContainerStarted","Data":"1572dd3a92f3f31add84734c72f62a2a3ca140738c623a94651a16d3989f95e2"} Jul 08 19:04:29.245118 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:29.245080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845c9d497b-sfzd8" event={"ID":"17653de6-6dde-4568-9b13-6504d200be01","Type":"ContainerStarted","Data":"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900"} Jul 08 19:04:29.265052 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:29.264965 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-845c9d497b-sfzd8" podStartSLOduration=1.5110370400000002 podStartE2EDuration="5.264948483s" totalImagesPullingTime="3.753911443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:29.263557341 +0000 UTC m=+325.691445652" watchObservedRunningTime="2026-07-08 19:04:29.264948483 +0000 UTC m=+325.692836791" Jul 08 19:04:31.253751 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:31.253719 2580 generic.go:356] "Generic (PLEG): container finished" podID="0b75b4e4-3ce0-418d-90ba-fd705325ff1f" containerID="7aff02ec7ab045e9e789dd7ca727f6a65ded8986ce4c0e10abfebea00487b721" exitCode=0 Jul 08 19:04:31.254092 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:31.253793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" event={"ID":"0b75b4e4-3ce0-418d-90ba-fd705325ff1f","Type":"ContainerDied","Data":"7aff02ec7ab045e9e789dd7ca727f6a65ded8986ce4c0e10abfebea00487b721"} Jul 08 19:04:31.254092 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:31.254082 2580 scope.go:117] "RemoveContainer" containerID="7aff02ec7ab045e9e789dd7ca727f6a65ded8986ce4c0e10abfebea00487b721" Jul 08 19:04:32.258446 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:32.258414 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-jzrb8" event={"ID":"0b75b4e4-3ce0-418d-90ba-fd705325ff1f","Type":"ContainerStarted","Data":"fec097b01fdfcf927805b45ecb2db2806f397e764aa15d3932440acb3492fb7c"} Jul 08 19:04:35.118612 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.118574 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:35.119014 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.118637 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:35.123345 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.123319 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:35.270568 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.270536 2580 generic.go:356] "Generic (PLEG): container finished" podID="a86510c6-35a5-48f4-badb-aea0921624d9" containerID="90721a677e09e53444603df45f2db71ed16744f152282cf68ccefa6843fa544f" exitCode=0 Jul 08 19:04:35.270714 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.270608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" event={"ID":"a86510c6-35a5-48f4-badb-aea0921624d9","Type":"ContainerDied","Data":"90721a677e09e53444603df45f2db71ed16744f152282cf68ccefa6843fa544f"} Jul 08 19:04:35.270978 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.270956 2580 scope.go:117] "RemoveContainer" containerID="90721a677e09e53444603df45f2db71ed16744f152282cf68ccefa6843fa544f" Jul 08 19:04:35.274614 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:35.274592 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:04:36.276102 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:36.276068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-llkzl" event={"ID":"a86510c6-35a5-48f4-badb-aea0921624d9","Type":"ContainerStarted","Data":"aede244964a9accadaae3fcb151f3f75a1142b6d4f592b2897dfe38fc70d1a68"} Jul 08 19:04:43.966223 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:43.966188 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cf956c5dc-r496w_125a062b-4f51-4040-bb38-f0a8a610d369/router/0.log" Jul 08 19:04:43.983090 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:04:43.983068 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n9mqf_b4a10718-e056-4a7a-851b-be327aff9606/serve-healthcheck-canary/0.log" Jul 08 19:05:18.010891 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.010837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:05:18.013164 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.013143 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7578c673-39a4-4942-9470-e85a33f20c0e-metrics-certs\") pod \"network-metrics-daemon-9gmnq\" (UID: \"7578c673-39a4-4942-9470-e85a33f20c0e\") " pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:05:18.111942 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.111915 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gvgdf\"" Jul 08 19:05:18.119404 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.119388 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gmnq" Jul 08 19:05:18.248225 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.248191 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gmnq"] Jul 08 19:05:18.250868 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:05:18.250844 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7578c673_39a4_4942_9470_e85a33f20c0e.slice/crio-069cf0e027f99a03747fce80ff9252272e317a3078dafeafcaf6f3f220f6bf3c WatchSource:0}: Error finding container 069cf0e027f99a03747fce80ff9252272e317a3078dafeafcaf6f3f220f6bf3c: Status 404 returned error can't find the container with id 069cf0e027f99a03747fce80ff9252272e317a3078dafeafcaf6f3f220f6bf3c Jul 08 19:05:18.439858 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:18.439778 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gmnq" event={"ID":"7578c673-39a4-4942-9470-e85a33f20c0e","Type":"ContainerStarted","Data":"069cf0e027f99a03747fce80ff9252272e317a3078dafeafcaf6f3f220f6bf3c"} Jul 08 19:05:19.445669 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:19.445591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gmnq" event={"ID":"7578c673-39a4-4942-9470-e85a33f20c0e","Type":"ContainerStarted","Data":"52efffe5e48ca34c00bf4f8f4951e013b981bb4318a608fb386f252111fddec5"} Jul 08 19:05:19.445669 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:19.445641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gmnq" event={"ID":"7578c673-39a4-4942-9470-e85a33f20c0e","Type":"ContainerStarted","Data":"131cb96b834f0170d8a50e3c39b48d2d151878074278ba1226602705417c4771"} Jul 08 19:05:19.464146 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:19.464098 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9gmnq" podStartSLOduration=374.55563683 podStartE2EDuration="6m15.464082624s" totalImagesPullingTime="908.445794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:19.462811208 +0000 UTC m=+375.890699515" watchObservedRunningTime="2026-07-08 19:05:19.464082624 +0000 UTC m=+375.891970932" Jul 08 19:05:45.846176 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:05:45.846143 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:06:10.866670 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:10.866610 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-845c9d497b-sfzd8" podUID="17653de6-6dde-4568-9b13-6504d200be01" containerName="console" containerID="cri-o://9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900" gracePeriod=15 Jul 08 19:06:11.120627 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.120563 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:06:11.183528 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183498 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183676 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jztz\" (UniqueName: \"kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183676 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183576 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183676 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183601 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183676 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183641 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183676 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183674 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.183922 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.183711 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config\") pod \"17653de6-6dde-4568-9b13-6504d200be01\" (UID: \"17653de6-6dde-4568-9b13-6504d200be01\") " Jul 08 19:06:11.184074 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.184042 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:11.184152 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.184070 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca" (OuterVolumeSpecName: "service-ca") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:11.184205 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.184142 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:11.184205 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.184148 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config" (OuterVolumeSpecName: "console-config") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:11.185889 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.185859 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz" (OuterVolumeSpecName: "kube-api-access-5jztz") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "kube-api-access-5jztz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:06:11.185889 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.185866 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:06:11.186019 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.185884 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "17653de6-6dde-4568-9b13-6504d200be01" (UID: "17653de6-6dde-4568-9b13-6504d200be01"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:06:11.285229 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285197 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-console-config\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285229 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285224 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-oauth-serving-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285229 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285234 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jztz\" (UniqueName: \"kubernetes.io/projected/17653de6-6dde-4568-9b13-6504d200be01-kube-api-access-5jztz\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285464 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285244 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-serving-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285464 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285252 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17653de6-6dde-4568-9b13-6504d200be01-console-oauth-config\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285464 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285261 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-service-ca\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.285464 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.285273 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17653de6-6dde-4568-9b13-6504d200be01-trusted-ca-bundle\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:06:11.644211 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.644172 2580 generic.go:356] "Generic (PLEG): container finished" podID="17653de6-6dde-4568-9b13-6504d200be01" containerID="9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900" exitCode=0 Jul 08 19:06:11.644392 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.644237 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-845c9d497b-sfzd8" Jul 08 19:06:11.644392 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.644253 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845c9d497b-sfzd8" event={"ID":"17653de6-6dde-4568-9b13-6504d200be01","Type":"ContainerDied","Data":"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900"} Jul 08 19:06:11.644392 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.644326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-845c9d497b-sfzd8" event={"ID":"17653de6-6dde-4568-9b13-6504d200be01","Type":"ContainerDied","Data":"1572dd3a92f3f31add84734c72f62a2a3ca140738c623a94651a16d3989f95e2"} Jul 08 19:06:11.644392 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.644351 2580 scope.go:117] "RemoveContainer" containerID="9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900" Jul 08 19:06:11.652437 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.652416 2580 scope.go:117] "RemoveContainer" containerID="9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900" Jul 08 19:06:11.652689 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:06:11.652671 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900\": container with ID starting with 9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900 not found: ID does not exist" containerID="9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900" Jul 08 19:06:11.652739 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.652698 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900"} err="failed to get container status \"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900\": rpc error: code = NotFound desc = could not find container \"9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900\": container with ID starting with 9551442fe9eb8feecb83ad23e8bcdec0d88d192b4f467fd2c0cacbca4658e900 not found: ID does not exist" Jul 08 19:06:11.670295 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.670273 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:06:11.673785 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:11.673764 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-845c9d497b-sfzd8"] Jul 08 19:06:12.213116 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:12.213084 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17653de6-6dde-4568-9b13-6504d200be01" path="/var/lib/kubelet/pods/17653de6-6dde-4568-9b13-6504d200be01/volumes" Jul 08 19:06:41.326150 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.326119 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:06:41.326605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.326417 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17653de6-6dde-4568-9b13-6504d200be01" containerName="console" Jul 08 19:06:41.326605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.326430 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17653de6-6dde-4568-9b13-6504d200be01" containerName="console" Jul 08 19:06:41.326605 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.326482 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17653de6-6dde-4568-9b13-6504d200be01" containerName="console" Jul 08 19:06:41.328352 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.328336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.330660 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.330637 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:06:41.331715 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.331645 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:06:41.331715 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.331654 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:06:41.331920 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.331710 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:06:41.332091 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.331983 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:06:41.332091 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.332011 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-659rr\"" Jul 08 19:06:41.332367 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.332162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 19:06:41.332813 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.332795 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 19:06:41.337315 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.337278 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:06:41.340525 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.340504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:06:41.400925 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.400903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401055 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.400938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401055 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.400978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401055 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.401044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401200 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.401090 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401200 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.401120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9d85\" (UniqueName: \"kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.401200 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.401178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502135 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9d85\" (UniqueName: \"kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502496 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502496 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502878 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502850 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502988 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502906 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.502988 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.502965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.503100 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.503084 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.504552 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.504529 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.504651 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.504635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.511693 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.511668 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9d85\" (UniqueName: \"kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85\") pod \"console-cf9f698df-t8bqq\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.637827 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.637765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:41.771204 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:41.771167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:06:41.774107 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:06:41.774083 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9332e3_b99e_4e51_be8c_c83f19529146.slice/crio-deb4f9d18cb8ece663702af1621763015befabaccc441cb739370c74a5dae18a WatchSource:0}: Error finding container deb4f9d18cb8ece663702af1621763015befabaccc441cb739370c74a5dae18a: Status 404 returned error can't find the container with id deb4f9d18cb8ece663702af1621763015befabaccc441cb739370c74a5dae18a Jul 08 19:06:42.760749 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:42.760713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf9f698df-t8bqq" event={"ID":"7d9332e3-b99e-4e51-be8c-c83f19529146","Type":"ContainerStarted","Data":"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40"} Jul 08 19:06:42.761133 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:42.760761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf9f698df-t8bqq" event={"ID":"7d9332e3-b99e-4e51-be8c-c83f19529146","Type":"ContainerStarted","Data":"deb4f9d18cb8ece663702af1621763015befabaccc441cb739370c74a5dae18a"} Jul 08 19:06:42.779844 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:42.779714 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cf9f698df-t8bqq" podStartSLOduration=1.7796929540000002 podStartE2EDuration="1.779692954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:42.779198539 +0000 UTC m=+459.207086841" watchObservedRunningTime="2026-07-08 19:06:42.779692954 +0000 UTC m=+459.207581262" Jul 08 19:06:51.638632 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:51.638592 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:51.639205 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:51.638679 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:51.643505 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:51.643483 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:06:51.796643 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:06:51.796614 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:08:25.839420 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.839334 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f4dvm"] Jul 08 19:08:25.841292 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.841273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:25.843852 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.843828 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 19:08:25.844968 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.844950 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 19:08:25.845055 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.845017 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wn24l\"" Jul 08 19:08:25.857018 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.856996 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f4dvm"] Jul 08 19:08:25.964073 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.964040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:25.964073 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:25.964081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dp6n\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-kube-api-access-4dp6n\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.064559 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.064529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.064683 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.064563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dp6n\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-kube-api-access-4dp6n\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.075116 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.075076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.075246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.075230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dp6n\" (UniqueName: \"kubernetes.io/projected/1b08050b-b424-4d95-9cda-00fb2e76df50-kube-api-access-4dp6n\") pod \"cert-manager-webhook-6486ffd997-f4dvm\" (UID: \"1b08050b-b424-4d95-9cda-00fb2e76df50\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.150202 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.150124 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:26.274997 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:26.274973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f4dvm"] Jul 08 19:08:26.277203 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:08:26.277172 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b08050b_b424_4d95_9cda_00fb2e76df50.slice/crio-ae0332adac65de838bd50022ad72f5b6fa7f8a1ad4dd080fa8d3e888bcf86e7d WatchSource:0}: Error finding container ae0332adac65de838bd50022ad72f5b6fa7f8a1ad4dd080fa8d3e888bcf86e7d: Status 404 returned error can't find the container with id ae0332adac65de838bd50022ad72f5b6fa7f8a1ad4dd080fa8d3e888bcf86e7d Jul 08 19:08:27.140429 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:27.140391 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" event={"ID":"1b08050b-b424-4d95-9cda-00fb2e76df50","Type":"ContainerStarted","Data":"ae0332adac65de838bd50022ad72f5b6fa7f8a1ad4dd080fa8d3e888bcf86e7d"} Jul 08 19:08:30.161146 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:30.161110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" event={"ID":"1b08050b-b424-4d95-9cda-00fb2e76df50","Type":"ContainerStarted","Data":"75f87eb9a58dda5cf2f4dffe56c798b056b1902ff41d4381497707e4095789c9"} Jul 08 19:08:30.161553 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:30.161172 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:30.178829 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:30.178785 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" podStartSLOduration=1.832566894 podStartE2EDuration="5.178771295s" totalImagesPullingTime="3.346204401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:30.177320651 +0000 UTC m=+566.605208959" watchObservedRunningTime="2026-07-08 19:08:30.178771295 +0000 UTC m=+566.606659603" Jul 08 19:08:36.168409 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:36.168377 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f4dvm" Jul 08 19:08:42.494897 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.492791 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664"] Jul 08 19:08:42.497119 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.497091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.500291 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.500271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 19:08:42.501448 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.501432 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:08:42.501506 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.501432 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-h7v7w\"" Jul 08 19:08:42.504824 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.504801 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664"] Jul 08 19:08:42.687413 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.687378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg69m\" (UniqueName: \"kubernetes.io/projected/93c53334-3f45-474f-879e-85f36ebfa324-kube-api-access-sg69m\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.687413 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.687412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c53334-3f45-474f-879e-85f36ebfa324-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.788134 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.788061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg69m\" (UniqueName: \"kubernetes.io/projected/93c53334-3f45-474f-879e-85f36ebfa324-kube-api-access-sg69m\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.788134 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.788096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c53334-3f45-474f-879e-85f36ebfa324-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.788458 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.788441 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93c53334-3f45-474f-879e-85f36ebfa324-tmp\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.797695 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.797675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg69m\" (UniqueName: \"kubernetes.io/projected/93c53334-3f45-474f-879e-85f36ebfa324-kube-api-access-sg69m\") pod \"openshift-lws-operator-fd8ccff4c-p9664\" (UID: \"93c53334-3f45-474f-879e-85f36ebfa324\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.806519 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.806502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" Jul 08 19:08:42.949460 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:42.949426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664"] Jul 08 19:08:42.953101 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:08:42.953077 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93c53334_3f45_474f_879e_85f36ebfa324.slice/crio-a439e77c591d0857b19f37f7bc70c2e66ee4b8745b0d1985841f03334994fa0f WatchSource:0}: Error finding container a439e77c591d0857b19f37f7bc70c2e66ee4b8745b0d1985841f03334994fa0f: Status 404 returned error can't find the container with id a439e77c591d0857b19f37f7bc70c2e66ee4b8745b0d1985841f03334994fa0f Jul 08 19:08:43.213148 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:43.213121 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" event={"ID":"93c53334-3f45-474f-879e-85f36ebfa324","Type":"ContainerStarted","Data":"a439e77c591d0857b19f37f7bc70c2e66ee4b8745b0d1985841f03334994fa0f"} Jul 08 19:08:46.227899 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:46.227863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" event={"ID":"93c53334-3f45-474f-879e-85f36ebfa324","Type":"ContainerStarted","Data":"cbb69f13485885b57c892b63716ad0a90ee64274e52d3b34b2eb493f6df712ee"} Jul 08 19:08:46.246134 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:08:46.246087 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-p9664" podStartSLOduration=1.651478175 podStartE2EDuration="4.246067486s" totalImagesPullingTime="2.594589311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:46.245666359 +0000 UTC m=+582.673554667" watchObservedRunningTime="2026-07-08 19:08:46.246067486 +0000 UTC m=+582.673955796" Jul 08 19:09:02.297186 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.297154 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn"] Jul 08 19:09:02.302829 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.302811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.308091 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.308015 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:09:02.308091 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.308067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:09:02.308374 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.308325 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:09:02.308374 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.308357 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:09:02.308767 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.308748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5zlwm\"" Jul 08 19:09:02.330633 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.330602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-tmp\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.330766 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.330686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.330829 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.330772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vwx5\" (UniqueName: \"kubernetes.io/projected/df3f5e05-9c06-4511-90c2-21dd0dd58469-kube-api-access-2vwx5\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.330829 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.330815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.330920 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.330858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-manifests\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.332361 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.332339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn"] Jul 08 19:09:02.431876 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.431845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-tmp\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.431876 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.431879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.432082 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.431961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vwx5\" (UniqueName: \"kubernetes.io/projected/df3f5e05-9c06-4511-90c2-21dd0dd58469-kube-api-access-2vwx5\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.432082 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.431986 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.432082 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.432012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-manifests\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.432438 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.432417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-manifests\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.434258 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.434236 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df3f5e05-9c06-4511-90c2-21dd0dd58469-tmp\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.434400 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.434383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.434586 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.434569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df3f5e05-9c06-4511-90c2-21dd0dd58469-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.454165 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.454145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vwx5\" (UniqueName: \"kubernetes.io/projected/df3f5e05-9c06-4511-90c2-21dd0dd58469-kube-api-access-2vwx5\") pod \"opendatahub-operator-controller-manager-7dd7775959-dbrvn\" (UID: \"df3f5e05-9c06-4511-90c2-21dd0dd58469\") " pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.613036 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.612953 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:02.790346 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:02.790315 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn"] Jul 08 19:09:02.790826 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:09:02.790801 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3f5e05_9c06_4511_90c2_21dd0dd58469.slice/crio-5fcd379ddde6047945ce84cb9af60ebdb78bd25db02ae7d2922eeb1946080fc1 WatchSource:0}: Error finding container 5fcd379ddde6047945ce84cb9af60ebdb78bd25db02ae7d2922eeb1946080fc1: Status 404 returned error can't find the container with id 5fcd379ddde6047945ce84cb9af60ebdb78bd25db02ae7d2922eeb1946080fc1 Jul 08 19:09:03.296396 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:03.296354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" event={"ID":"df3f5e05-9c06-4511-90c2-21dd0dd58469","Type":"ContainerStarted","Data":"5fcd379ddde6047945ce84cb9af60ebdb78bd25db02ae7d2922eeb1946080fc1"} Jul 08 19:09:04.048825 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:04.048794 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:09:04.049410 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:04.049384 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:09:06.312091 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:06.311991 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" event={"ID":"df3f5e05-9c06-4511-90c2-21dd0dd58469","Type":"ContainerStarted","Data":"bbd069ab517cb97347c6b2068810a8235dde167d97d8ea79a9ec7c0a4471e8ac"} Jul 08 19:09:07.317646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:07.317556 2580 generic.go:356] "Generic (PLEG): container finished" podID="df3f5e05-9c06-4511-90c2-21dd0dd58469" containerID="bbd069ab517cb97347c6b2068810a8235dde167d97d8ea79a9ec7c0a4471e8ac" exitCode=0 Jul 08 19:09:07.317646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:07.317604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" event={"ID":"df3f5e05-9c06-4511-90c2-21dd0dd58469","Type":"ContainerDied","Data":"bbd069ab517cb97347c6b2068810a8235dde167d97d8ea79a9ec7c0a4471e8ac"} Jul 08 19:09:07.318129 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:07.318114 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:09:08.323589 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:08.323553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" event={"ID":"df3f5e05-9c06-4511-90c2-21dd0dd58469","Type":"ContainerStarted","Data":"8492c85998dba5623b9a68b6bcd57cfacdf3afcef6ca8ab2c52c1345f7dd976f"} Jul 08 19:09:08.324068 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:08.323678 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:08.359679 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:08.359631 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" podStartSLOduration=1.8984790280000001 podStartE2EDuration="6.359615728s" totalImagesPullingTime="3.379200266s" totalInitContainerRuntime="1.081936434s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:08.357309854 +0000 UTC m=+604.785198154" watchObservedRunningTime="2026-07-08 19:09:08.359615728 +0000 UTC m=+604.787504082" Jul 08 19:09:19.329685 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:19.329656 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dd7775959-dbrvn" Jul 08 19:09:22.372094 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.372059 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c"] Jul 08 19:09:22.386267 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.386242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.389354 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.389335 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 19:09:22.389354 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.389342 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 19:09:22.389627 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.389612 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7jkxs\"" Jul 08 19:09:22.393997 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.393977 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c"] Jul 08 19:09:22.483492 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.483454 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffl8"] Jul 08 19:09:22.486805 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.486788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:22.489660 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.489641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 19:09:22.489713 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.489642 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wvkbh\"" Jul 08 19:09:22.499881 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.499856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.500012 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.499896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tmp\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.500012 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.499983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94djr\" (UniqueName: \"kubernetes.io/projected/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-kube-api-access-94djr\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.500167 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.500150 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffl8"] Jul 08 19:09:22.601031 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.600998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wbw8\" (UniqueName: \"kubernetes.io/projected/d3d98d42-4048-4c2a-9e6b-2c10b980d744-kube-api-access-6wbw8\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:22.601188 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.601057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.601188 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.601111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tmp\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.601188 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:09:22.601150 2580 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 08 19:09:22.601188 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.601163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94djr\" (UniqueName: \"kubernetes.io/projected/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-kube-api-access-94djr\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.601188 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.601185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:22.601443 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:09:22.601223 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs podName:f107e6dd-9b2b-42a4-b186-04b4b2f778a5 nodeName:}" failed. No retries permitted until 2026-07-08 19:09:23.101207045 +0000 UTC m=+619.529095331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs") pod "kube-auth-proxy-85bd9454df-q9k5c" (UID: "f107e6dd-9b2b-42a4-b186-04b4b2f778a5") : secret "kube-auth-proxy-tls" not found Jul 08 19:09:22.603311 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.603284 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tmp\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.612094 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.612075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94djr\" (UniqueName: \"kubernetes.io/projected/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-kube-api-access-94djr\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:22.702601 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.702516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:22.702601 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.702568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wbw8\" (UniqueName: \"kubernetes.io/projected/d3d98d42-4048-4c2a-9e6b-2c10b980d744-kube-api-access-6wbw8\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:22.702828 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:09:22.702679 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 19:09:22.702828 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:09:22.702764 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert podName:d3d98d42-4048-4c2a-9e6b-2c10b980d744 nodeName:}" failed. No retries permitted until 2026-07-08 19:09:23.202744282 +0000 UTC m=+619.630632568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert") pod "odh-model-controller-7998c67dc4-pffl8" (UID: "d3d98d42-4048-4c2a-9e6b-2c10b980d744") : secret "odh-model-controller-webhook-cert" not found Jul 08 19:09:22.716241 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:22.716217 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wbw8\" (UniqueName: \"kubernetes.io/projected/d3d98d42-4048-4c2a-9e6b-2c10b980d744-kube-api-access-6wbw8\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:23.105689 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.105645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:23.107979 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.107957 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f107e6dd-9b2b-42a4-b186-04b4b2f778a5-tls-certs\") pod \"kube-auth-proxy-85bd9454df-q9k5c\" (UID: \"f107e6dd-9b2b-42a4-b186-04b4b2f778a5\") " pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:23.206618 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.206580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:23.209232 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.209203 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3d98d42-4048-4c2a-9e6b-2c10b980d744-cert\") pod \"odh-model-controller-7998c67dc4-pffl8\" (UID: \"d3d98d42-4048-4c2a-9e6b-2c10b980d744\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:23.296146 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.296112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" Jul 08 19:09:23.396663 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.396634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:23.516659 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.516623 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c"] Jul 08 19:09:23.520769 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:09:23.520737 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf107e6dd_9b2b_42a4_b186_04b4b2f778a5.slice/crio-6cf38bfa21f189790fdee4b8916a2a6da45557ae71eca21517ff60dd43baf7e9 WatchSource:0}: Error finding container 6cf38bfa21f189790fdee4b8916a2a6da45557ae71eca21517ff60dd43baf7e9: Status 404 returned error can't find the container with id 6cf38bfa21f189790fdee4b8916a2a6da45557ae71eca21517ff60dd43baf7e9 Jul 08 19:09:23.552918 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:23.552892 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffl8"] Jul 08 19:09:23.554809 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:09:23.554781 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d98d42_4048_4c2a_9e6b_2c10b980d744.slice/crio-57639f7c5d4e017a30c82bef7a351704582e9788ad5cb50e21285deda5a4d463 WatchSource:0}: Error finding container 57639f7c5d4e017a30c82bef7a351704582e9788ad5cb50e21285deda5a4d463: Status 404 returned error can't find the container with id 57639f7c5d4e017a30c82bef7a351704582e9788ad5cb50e21285deda5a4d463 Jul 08 19:09:24.395540 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.395499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" event={"ID":"f107e6dd-9b2b-42a4-b186-04b4b2f778a5","Type":"ContainerStarted","Data":"6cf38bfa21f189790fdee4b8916a2a6da45557ae71eca21517ff60dd43baf7e9"} Jul 08 19:09:24.397004 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.396965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" event={"ID":"d3d98d42-4048-4c2a-9e6b-2c10b980d744","Type":"ContainerStarted","Data":"57639f7c5d4e017a30c82bef7a351704582e9788ad5cb50e21285deda5a4d463"} Jul 08 19:09:24.423202 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.423166 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:24.426508 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.426487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.437746 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.437715 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:09:24.437856 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.437747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:09:24.438645 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.438624 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:09:24.438645 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.438637 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gj98c\"" Jul 08 19:09:24.438834 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.438629 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:09:24.459132 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.459107 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:24.519097 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519097 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5lsq\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519358 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.519572 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.519420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620161 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5lsq\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620380 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620380 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620380 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620380 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620355 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620574 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.620574 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.620432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.621163 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.621132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.623447 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.623422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.624320 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.624252 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.624638 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.624612 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.624775 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.624756 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.630036 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.630011 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.632418 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.632389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5lsq\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq\") pod \"istiod-openshift-gateway-7574459656-znqcq\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.738746 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.738650 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:24.899931 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:24.899906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:24.902409 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:09:24.902379 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3668fe_2ac9_4fa4_ae02_2f46428771fa.slice/crio-f72dbcb527f3c1b27c0fa1dc7265fd6e6015bef328b090f0f9edced9d9b36bac WatchSource:0}: Error finding container f72dbcb527f3c1b27c0fa1dc7265fd6e6015bef328b090f0f9edced9d9b36bac: Status 404 returned error can't find the container with id f72dbcb527f3c1b27c0fa1dc7265fd6e6015bef328b090f0f9edced9d9b36bac Jul 08 19:09:25.405704 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:25.405291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" event={"ID":"fa3668fe-2ac9-4fa4-ae02-2f46428771fa","Type":"ContainerStarted","Data":"f72dbcb527f3c1b27c0fa1dc7265fd6e6015bef328b090f0f9edced9d9b36bac"} Jul 08 19:09:28.432259 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:28.432220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" event={"ID":"d3d98d42-4048-4c2a-9e6b-2c10b980d744","Type":"ContainerStarted","Data":"a4dd066ab962dc6af0c32b5f478f17ce39170e5134ab205b619649d451725e79"} Jul 08 19:09:28.432764 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:28.432367 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:28.434620 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:28.434586 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" event={"ID":"f107e6dd-9b2b-42a4-b186-04b4b2f778a5","Type":"ContainerStarted","Data":"a9982a069ac6920706ca9299bfb6712289a267289109a5f2a65423473bf24c0d"} Jul 08 19:09:28.450755 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:28.450709 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" podStartSLOduration=2.171121592 podStartE2EDuration="6.450694957s" totalImagesPullingTime="4.279573365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:28.450223862 +0000 UTC m=+624.878112170" watchObservedRunningTime="2026-07-08 19:09:28.450694957 +0000 UTC m=+624.878583326" Jul 08 19:09:28.467064 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:28.467016 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85bd9454df-q9k5c" podStartSLOduration=2.158911508 podStartE2EDuration="6.46700336s" totalImagesPullingTime="4.308091852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:28.466080013 +0000 UTC m=+624.893968318" watchObservedRunningTime="2026-07-08 19:09:28.46700336 +0000 UTC m=+624.894891708" Jul 08 19:09:32.373623 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.373591 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4"] Jul 08 19:09:32.377417 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.377396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.385354 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.385248 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 19:09:32.385651 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.385621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q9mlg\"" Jul 08 19:09:32.386617 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.386599 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 19:09:32.392937 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.392919 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 19:09:32.454412 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.454378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4"] Jul 08 19:09:32.495581 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.495539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96c4e6df-17ed-4e81-b6d4-505ad41438ba-manager-config\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.495737 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.495594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.495737 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.495660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb7l7\" (UniqueName: \"kubernetes.io/projected/96c4e6df-17ed-4e81-b6d4-505ad41438ba-kube-api-access-zb7l7\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.495839 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.495748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-metrics-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.596854 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.596812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-metrics-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.597045 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.596906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96c4e6df-17ed-4e81-b6d4-505ad41438ba-manager-config\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.597045 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.596945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.597045 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.596981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb7l7\" (UniqueName: \"kubernetes.io/projected/96c4e6df-17ed-4e81-b6d4-505ad41438ba-kube-api-access-zb7l7\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.597672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.597646 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96c4e6df-17ed-4e81-b6d4-505ad41438ba-manager-config\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.599686 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.599659 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.599974 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.599942 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96c4e6df-17ed-4e81-b6d4-505ad41438ba-metrics-cert\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.610723 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.610695 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb7l7\" (UniqueName: \"kubernetes.io/projected/96c4e6df-17ed-4e81-b6d4-505ad41438ba-kube-api-access-zb7l7\") pod \"lws-controller-manager-5bfc697457-ckzq4\" (UID: \"96c4e6df-17ed-4e81-b6d4-505ad41438ba\") " pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:32.688483 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:32.688387 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:33.492031 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:33.491982 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 19:09:33.492350 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:33.492078 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 08 19:09:33.580526 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:33.580503 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4"] Jul 08 19:09:33.582143 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:09:33.582116 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c4e6df_17ed_4e81_b6d4_505ad41438ba.slice/crio-c74316889331608edf956a642990f2135df0c096aecf29ea36252ab5f14024ad WatchSource:0}: Error finding container c74316889331608edf956a642990f2135df0c096aecf29ea36252ab5f14024ad: Status 404 returned error can't find the container with id c74316889331608edf956a642990f2135df0c096aecf29ea36252ab5f14024ad Jul 08 19:09:34.468357 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.468290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" event={"ID":"fa3668fe-2ac9-4fa4-ae02-2f46428771fa","Type":"ContainerStarted","Data":"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d"} Jul 08 19:09:34.468538 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.468489 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:34.469745 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.469709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" event={"ID":"96c4e6df-17ed-4e81-b6d4-505ad41438ba","Type":"ContainerStarted","Data":"c74316889331608edf956a642990f2135df0c096aecf29ea36252ab5f14024ad"} Jul 08 19:09:34.469995 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.469947 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-znqcq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:09:34.470103 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.470039 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:09:34.534407 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:34.534342 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" podStartSLOduration=1.947052036 podStartE2EDuration="10.534321399s" totalImagesPullingTime="8.587269363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:34.533737662 +0000 UTC m=+630.961625973" watchObservedRunningTime="2026-07-08 19:09:34.534321399 +0000 UTC m=+630.962209711" Jul 08 19:09:35.475793 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:35.475756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" event={"ID":"96c4e6df-17ed-4e81-b6d4-505ad41438ba","Type":"ContainerStarted","Data":"b8d073c2828f149e3ed17b953597aaccdea628057dc21b3d7afac6ab1cb31853"} Jul 08 19:09:35.476081 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:35.476057 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:09:35.476396 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:35.476368 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-znqcq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:09:35.476514 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:35.476427 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:09:35.503533 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:35.503485 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" podStartSLOduration=2.077359518 podStartE2EDuration="3.503464067s" totalImagesPullingTime="1.426104549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:35.501537307 +0000 UTC m=+631.929425615" watchObservedRunningTime="2026-07-08 19:09:35.503464067 +0000 UTC m=+631.931352442" Jul 08 19:09:37.790592 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:37.790550 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:37.791067 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:37.790760 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" containerID="cri-o://83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d" gracePeriod=30 Jul 08 19:09:37.791538 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:37.791473 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:38.044138 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.044086 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:38.143670 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143640 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.143821 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143676 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.143821 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.143821 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143737 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.143821 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143770 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.143821 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143811 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.144074 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.143835 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5lsq\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq\") pod \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\" (UID: \"fa3668fe-2ac9-4fa4-ae02-2f46428771fa\") " Jul 08 19:09:38.144252 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.144227 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:09:38.146096 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146068 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs" (OuterVolumeSpecName: "local-certs") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:09:38.146197 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146120 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq" (OuterVolumeSpecName: "kube-api-access-w5lsq") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "kube-api-access-w5lsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:09:38.146269 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146197 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts" (OuterVolumeSpecName: "cacerts") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:09:38.146269 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146259 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:09:38.146400 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146284 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:09:38.146400 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.146315 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fa3668fe-2ac9-4fa4-ae02-2f46428771fa" (UID: "fa3668fe-2ac9-4fa4-ae02-2f46428771fa"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:09:38.245110 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245077 2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-local-certs\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245113 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-token\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245128 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-ca-configmap\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245141 2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-kubeconfig\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245151 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5lsq\" (UniqueName: \"kubernetes.io/projected/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-kube-api-access-w5lsq\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245159 2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-cacerts\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.245226 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.245167 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fa3668fe-2ac9-4fa4-ae02-2f46428771fa-istio-csr-dns-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:09:38.490176 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.490136 2580 generic.go:356] "Generic (PLEG): container finished" podID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerID="83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d" exitCode=0 Jul 08 19:09:38.490331 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.490190 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" Jul 08 19:09:38.490331 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.490227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" event={"ID":"fa3668fe-2ac9-4fa4-ae02-2f46428771fa","Type":"ContainerDied","Data":"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d"} Jul 08 19:09:38.490331 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.490291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-znqcq" event={"ID":"fa3668fe-2ac9-4fa4-ae02-2f46428771fa","Type":"ContainerDied","Data":"f72dbcb527f3c1b27c0fa1dc7265fd6e6015bef328b090f0f9edced9d9b36bac"} Jul 08 19:09:38.490470 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.490334 2580 scope.go:117] "RemoveContainer" containerID="83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d" Jul 08 19:09:38.498427 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.498406 2580 scope.go:117] "RemoveContainer" containerID="83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d" Jul 08 19:09:38.498682 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:09:38.498664 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d\": container with ID starting with 83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d not found: ID does not exist" containerID="83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d" Jul 08 19:09:38.498734 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.498688 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d"} err="failed to get container status \"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d\": rpc error: code = NotFound desc = could not find container \"83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d\": container with ID starting with 83bdbb98a5aed9a9588be1cc336dbf3d66354710833e3daf5ac3e94e57b25d0d not found: ID does not exist" Jul 08 19:09:38.533470 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.533446 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:38.535212 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:38.535190 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-znqcq"] Jul 08 19:09:39.441392 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:39.441361 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pffl8" Jul 08 19:09:40.213607 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:40.213575 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" path="/var/lib/kubelet/pods/fa3668fe-2ac9-4fa4-ae02-2f46428771fa/volumes" Jul 08 19:09:46.482363 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:09:46.482331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bfc697457-ckzq4" Jul 08 19:11:01.140308 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:01.140219 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:11:14.569181 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.569145 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl"] Jul 08 19:11:14.569837 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.569640 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" Jul 08 19:11:14.569837 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.569656 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" Jul 08 19:11:14.569837 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.569749 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa3668fe-2ac9-4fa4-ae02-2f46428771fa" containerName="discovery" Jul 08 19:11:14.572732 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.572713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.575275 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.575240 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 19:11:14.575406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.575245 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 19:11:14.575406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.575331 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lhrps\"" Jul 08 19:11:14.575406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.575359 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:11:14.576260 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.576243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:11:14.583285 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.583265 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl"] Jul 08 19:11:14.721423 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.721391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de39805-9ccd-439b-beff-4361c91988cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.721599 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.721443 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5de39805-9ccd-439b-beff-4361c91988cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.721599 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.721527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hggtt\" (UniqueName: \"kubernetes.io/projected/5de39805-9ccd-439b-beff-4361c91988cf-kube-api-access-hggtt\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.822171 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.822085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de39805-9ccd-439b-beff-4361c91988cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.822171 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.822153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5de39805-9ccd-439b-beff-4361c91988cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.822396 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.822186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hggtt\" (UniqueName: \"kubernetes.io/projected/5de39805-9ccd-439b-beff-4361c91988cf-kube-api-access-hggtt\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.822908 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.822885 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5de39805-9ccd-439b-beff-4361c91988cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.824546 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.824527 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5de39805-9ccd-439b-beff-4361c91988cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.832592 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.832570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hggtt\" (UniqueName: \"kubernetes.io/projected/5de39805-9ccd-439b-beff-4361c91988cf-kube-api-access-hggtt\") pod \"kuadrant-console-plugin-66b5774985-9zjgl\" (UID: \"5de39805-9ccd-439b-beff-4361c91988cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:14.881789 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:14.881765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" Jul 08 19:11:15.014665 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:15.014633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl"] Jul 08 19:11:15.018689 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:11:15.018663 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5de39805_9ccd_439b_beff_4361c91988cf.slice/crio-b5a4605d61ce564d16336a07bfbf0c211fbf55b9835b9def9f5278f8a81de106 WatchSource:0}: Error finding container b5a4605d61ce564d16336a07bfbf0c211fbf55b9835b9def9f5278f8a81de106: Status 404 returned error can't find the container with id b5a4605d61ce564d16336a07bfbf0c211fbf55b9835b9def9f5278f8a81de106 Jul 08 19:11:15.921026 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:15.920983 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" event={"ID":"5de39805-9ccd-439b-beff-4361c91988cf","Type":"ContainerStarted","Data":"b5a4605d61ce564d16336a07bfbf0c211fbf55b9835b9def9f5278f8a81de106"} Jul 08 19:11:26.160463 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:26.160424 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cf9f698df-t8bqq" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerName="console" containerID="cri-o://091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40" gracePeriod=15 Jul 08 19:11:31.793427 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:31.793385 2580 patch_prober.go:28] interesting pod/console-cf9f698df-t8bqq container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jul 08 19:11:31.793876 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:31.793472 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-cf9f698df-t8bqq" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jul 08 19:11:40.419348 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.419316 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:11:40.442631 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442605 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.442757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442660 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.442757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442696 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.442757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442734 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442789 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442843 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9d85\" (UniqueName: \"kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442869 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert\") pod \"7d9332e3-b99e-4e51-be8c-c83f19529146\" (UID: \"7d9332e3-b99e-4e51-be8c-c83f19529146\") " Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.442934 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config" (OuterVolumeSpecName: "console-config") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.443213 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-console-config\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.443249 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:11:40.443550 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.443498 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca" (OuterVolumeSpecName: "service-ca") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:11:40.443913 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.443869 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:11:40.445321 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.445271 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:11:40.446070 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.446042 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85" (OuterVolumeSpecName: "kube-api-access-b9d85") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "kube-api-access-b9d85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:11:40.446360 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.446338 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7d9332e3-b99e-4e51-be8c-c83f19529146" (UID: "7d9332e3-b99e-4e51-be8c-c83f19529146"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:11:40.544138 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544113 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9d85\" (UniqueName: \"kubernetes.io/projected/7d9332e3-b99e-4e51-be8c-c83f19529146-kube-api-access-b9d85\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.544138 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544135 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-oauth-serving-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.544282 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544145 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-service-ca\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.544282 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544153 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-serving-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.544282 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544162 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d9332e3-b99e-4e51-be8c-c83f19529146-console-oauth-config\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:40.544282 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:40.544169 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d9332e3-b99e-4e51-be8c-c83f19529146-trusted-ca-bundle\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:11:41.051475 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.051440 2580 generic.go:356] "Generic (PLEG): container finished" podID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerID="091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40" exitCode=0 Jul 08 19:11:41.051691 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.051480 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf9f698df-t8bqq" Jul 08 19:11:41.051691 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.051523 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf9f698df-t8bqq" event={"ID":"7d9332e3-b99e-4e51-be8c-c83f19529146","Type":"ContainerDied","Data":"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40"} Jul 08 19:11:41.051691 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.051578 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf9f698df-t8bqq" event={"ID":"7d9332e3-b99e-4e51-be8c-c83f19529146","Type":"ContainerDied","Data":"deb4f9d18cb8ece663702af1621763015befabaccc441cb739370c74a5dae18a"} Jul 08 19:11:41.051691 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.051608 2580 scope.go:117] "RemoveContainer" containerID="091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40" Jul 08 19:11:41.053754 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.053728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" event={"ID":"5de39805-9ccd-439b-beff-4361c91988cf","Type":"ContainerStarted","Data":"0e183c4652c93b153e2d3b0237ccedcff2198354158dd014170a7a205795ca59"} Jul 08 19:11:41.060493 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.060473 2580 scope.go:117] "RemoveContainer" containerID="091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40" Jul 08 19:11:41.060738 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:11:41.060719 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40\": container with ID starting with 091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40 not found: ID does not exist" containerID="091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40" Jul 08 19:11:41.060802 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.060750 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40"} err="failed to get container status \"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40\": rpc error: code = NotFound desc = could not find container \"091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40\": container with ID starting with 091dbdbd0763702c0cb86dc2984d597a35a818eac4d513fd02bafec90c25ae40 not found: ID does not exist" Jul 08 19:11:41.072717 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.072680 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9zjgl" podStartSLOduration=1.657823453 podStartE2EDuration="27.072668954s" totalImagesPullingTime="25.414845501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:41.07145417 +0000 UTC m=+757.499342478" watchObservedRunningTime="2026-07-08 19:11:41.072668954 +0000 UTC m=+757.500557261" Jul 08 19:11:41.086216 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.086194 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:11:41.097934 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:41.097913 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cf9f698df-t8bqq"] Jul 08 19:11:42.220402 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:42.220368 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" path="/var/lib/kubelet/pods/7d9332e3-b99e-4e51-be8c-c83f19529146/volumes" Jul 08 19:11:55.726261 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.726225 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:11:55.726745 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.726592 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerName="console" Jul 08 19:11:55.726745 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.726603 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerName="console" Jul 08 19:11:55.726745 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.726673 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d9332e3-b99e-4e51-be8c-c83f19529146" containerName="console" Jul 08 19:11:55.761796 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.761771 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:11:55.761796 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.761805 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:11:55.761973 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.761882 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.764830 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.764802 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 19:11:55.867265 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.867232 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-config-file\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.867420 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.867358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg8cp\" (UniqueName: \"kubernetes.io/projected/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-kube-api-access-xg8cp\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.967820 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.967791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg8cp\" (UniqueName: \"kubernetes.io/projected/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-kube-api-access-xg8cp\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.967971 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.967829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-config-file\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.968554 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.968533 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-config-file\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:55.976403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:55.976362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg8cp\" (UniqueName: \"kubernetes.io/projected/a1d0bcb7-f160-441d-9c83-67d8de69ad9a-kube-api-access-xg8cp\") pod \"limitador-limitador-54d6d988d5-b4ctp\" (UID: \"a1d0bcb7-f160-441d-9c83-67d8de69ad9a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:56.072268 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.072227 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:56.150595 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.150554 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:11:56.153560 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.153540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:11:56.155999 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.155977 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r57js\"" Jul 08 19:11:56.162846 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.162796 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:11:56.213619 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:11:56.213590 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d0bcb7_f160_441d_9c83_67d8de69ad9a.slice/crio-5b75adb4b79a02c96d4e9d56b341cc06005aed6919492282993fc546dbfde486 WatchSource:0}: Error finding container 5b75adb4b79a02c96d4e9d56b341cc06005aed6919492282993fc546dbfde486: Status 404 returned error can't find the container with id 5b75adb4b79a02c96d4e9d56b341cc06005aed6919492282993fc546dbfde486 Jul 08 19:11:56.214951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.214929 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:11:56.269480 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.269411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftjbb\" (UniqueName: \"kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb\") pod \"authorino-58cd955675-hkm9z\" (UID: \"2e041f23-655b-47ee-b7bc-a1fe23a450ba\") " pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:11:56.370519 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.370482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftjbb\" (UniqueName: \"kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb\") pod \"authorino-58cd955675-hkm9z\" (UID: \"2e041f23-655b-47ee-b7bc-a1fe23a450ba\") " pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:11:56.379751 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.379723 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftjbb\" (UniqueName: \"kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb\") pod \"authorino-58cd955675-hkm9z\" (UID: \"2e041f23-655b-47ee-b7bc-a1fe23a450ba\") " pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:11:56.466753 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.466722 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:11:56.591318 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:56.591269 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:11:56.594688 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:11:56.594664 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e041f23_655b_47ee_b7bc_a1fe23a450ba.slice/crio-449e6c45a2a3ec8c77a833e2c94b6d2f210fa71cba6bfe025b444721e8a30483 WatchSource:0}: Error finding container 449e6c45a2a3ec8c77a833e2c94b6d2f210fa71cba6bfe025b444721e8a30483: Status 404 returned error can't find the container with id 449e6c45a2a3ec8c77a833e2c94b6d2f210fa71cba6bfe025b444721e8a30483 Jul 08 19:11:57.131339 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:57.131279 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" event={"ID":"a1d0bcb7-f160-441d-9c83-67d8de69ad9a","Type":"ContainerStarted","Data":"5b75adb4b79a02c96d4e9d56b341cc06005aed6919492282993fc546dbfde486"} Jul 08 19:11:57.132694 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:57.132664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hkm9z" event={"ID":"2e041f23-655b-47ee-b7bc-a1fe23a450ba","Type":"ContainerStarted","Data":"449e6c45a2a3ec8c77a833e2c94b6d2f210fa71cba6bfe025b444721e8a30483"} Jul 08 19:11:59.148294 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:59.148249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" event={"ID":"a1d0bcb7-f160-441d-9c83-67d8de69ad9a","Type":"ContainerStarted","Data":"aa2b13b6f05e287f4ea16e9e5cbd50e34a233b3b95b7be7e230a68ab6e835b0d"} Jul 08 19:11:59.148902 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:59.148353 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:11:59.167471 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:11:59.167421 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" podStartSLOduration=1.4230479759999999 podStartE2EDuration="4.167402424s" totalImagesPullingTime="2.744354448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:59.165160861 +0000 UTC m=+775.593049170" watchObservedRunningTime="2026-07-08 19:11:59.167402424 +0000 UTC m=+775.595290733" Jul 08 19:12:02.163856 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:02.163811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hkm9z" event={"ID":"2e041f23-655b-47ee-b7bc-a1fe23a450ba","Type":"ContainerStarted","Data":"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479"} Jul 08 19:12:02.181294 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:02.181242 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hkm9z" podStartSLOduration=1.632567201 podStartE2EDuration="6.181226901s" totalImagesPullingTime="4.5486597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:02.180268583 +0000 UTC m=+778.608156893" watchObservedRunningTime="2026-07-08 19:12:02.181226901 +0000 UTC m=+778.609115209" Jul 08 19:12:10.153782 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:10.153752 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4ctp" Jul 08 19:12:22.546488 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:22.546448 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:12:22.546998 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:22.546671 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hkm9z" podUID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" containerName="authorino" containerID="cri-o://dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479" gracePeriod=30 Jul 08 19:12:22.797174 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:22.797119 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:12:22.902403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:22.902369 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftjbb\" (UniqueName: \"kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb\") pod \"2e041f23-655b-47ee-b7bc-a1fe23a450ba\" (UID: \"2e041f23-655b-47ee-b7bc-a1fe23a450ba\") " Jul 08 19:12:22.904505 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:22.904478 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb" (OuterVolumeSpecName: "kube-api-access-ftjbb") pod "2e041f23-655b-47ee-b7bc-a1fe23a450ba" (UID: "2e041f23-655b-47ee-b7bc-a1fe23a450ba"). InnerVolumeSpecName "kube-api-access-ftjbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:23.003412 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.003385 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftjbb\" (UniqueName: \"kubernetes.io/projected/2e041f23-655b-47ee-b7bc-a1fe23a450ba-kube-api-access-ftjbb\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:12:23.251489 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.251454 2580 generic.go:356] "Generic (PLEG): container finished" podID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" containerID="dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479" exitCode=0 Jul 08 19:12:23.251664 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.251496 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hkm9z" Jul 08 19:12:23.251664 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.251543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hkm9z" event={"ID":"2e041f23-655b-47ee-b7bc-a1fe23a450ba","Type":"ContainerDied","Data":"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479"} Jul 08 19:12:23.251664 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.251589 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hkm9z" event={"ID":"2e041f23-655b-47ee-b7bc-a1fe23a450ba","Type":"ContainerDied","Data":"449e6c45a2a3ec8c77a833e2c94b6d2f210fa71cba6bfe025b444721e8a30483"} Jul 08 19:12:23.251664 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.251611 2580 scope.go:117] "RemoveContainer" containerID="dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479" Jul 08 19:12:23.260096 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.260076 2580 scope.go:117] "RemoveContainer" containerID="dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479" Jul 08 19:12:23.260399 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:12:23.260382 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479\": container with ID starting with dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479 not found: ID does not exist" containerID="dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479" Jul 08 19:12:23.260446 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.260411 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479"} err="failed to get container status \"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479\": rpc error: code = NotFound desc = could not find container \"dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479\": container with ID starting with dd673ec3e4f4cbea9cc5203c87411d4a7542cc9d90f0a92ff23b656572286479 not found: ID does not exist" Jul 08 19:12:23.273501 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.273478 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:12:23.276787 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:23.276759 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hkm9z"] Jul 08 19:12:24.213463 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:24.213427 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" path="/var/lib/kubelet/pods/2e041f23-655b-47ee-b7bc-a1fe23a450ba/volumes" Jul 08 19:12:35.114988 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.114955 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:35.115465 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.115324 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" containerName="authorino" Jul 08 19:12:35.115465 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.115337 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" containerName="authorino" Jul 08 19:12:35.115465 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.115398 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e041f23-655b-47ee-b7bc-a1fe23a450ba" containerName="authorino" Jul 08 19:12:35.158374 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.158342 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:35.158531 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.158456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:35.160952 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.160934 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r57js\"" Jul 08 19:12:35.198591 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.198555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-999ff\" (UniqueName: \"kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff\") pod \"authorino-5d4b67c446-rxjl9\" (UID: \"8ceeb7ab-9c1b-48dd-b326-acf896770a6f\") " pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:35.299006 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.298972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-999ff\" (UniqueName: \"kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff\") pod \"authorino-5d4b67c446-rxjl9\" (UID: \"8ceeb7ab-9c1b-48dd-b326-acf896770a6f\") " pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:35.307504 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.307480 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-999ff\" (UniqueName: \"kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff\") pod \"authorino-5d4b67c446-rxjl9\" (UID: \"8ceeb7ab-9c1b-48dd-b326-acf896770a6f\") " pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:35.350825 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.350794 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:35.351056 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.351041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:35.374962 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.374905 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b54745cf5-vtx5h"] Jul 08 19:12:35.377153 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.377134 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:35.384941 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.384914 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b54745cf5-vtx5h"] Jul 08 19:12:35.399816 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.399788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74gsf\" (UniqueName: \"kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf\") pod \"authorino-5b54745cf5-vtx5h\" (UID: \"957e1b1d-8e2f-4a92-aec9-a7911122ae71\") " pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:35.452475 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.452442 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b54745cf5-vtx5h"] Jul 08 19:12:35.452800 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:12:35.452746 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-74gsf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5b54745cf5-vtx5h" podUID="957e1b1d-8e2f-4a92-aec9-a7911122ae71" Jul 08 19:12:35.480077 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.480050 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:12:35.482348 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.482322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.484792 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.484774 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 19:12:35.484916 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.484890 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:35.487785 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:12:35.487764 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ceeb7ab_9c1b_48dd_b326_acf896770a6f.slice/crio-f045c0423397266fc9cf4291df6dc1d605d4dcf02a54025d9d09079b87249d12 WatchSource:0}: Error finding container f045c0423397266fc9cf4291df6dc1d605d4dcf02a54025d9d09079b87249d12: Status 404 returned error can't find the container with id f045c0423397266fc9cf4291df6dc1d605d4dcf02a54025d9d09079b87249d12 Jul 08 19:12:35.493943 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.493900 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:12:35.500418 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.500396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnpml\" (UniqueName: \"kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.500502 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.500430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.500549 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.500520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74gsf\" (UniqueName: \"kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf\") pod \"authorino-5b54745cf5-vtx5h\" (UID: \"957e1b1d-8e2f-4a92-aec9-a7911122ae71\") " pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:35.508472 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.508454 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74gsf\" (UniqueName: \"kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf\") pod \"authorino-5b54745cf5-vtx5h\" (UID: \"957e1b1d-8e2f-4a92-aec9-a7911122ae71\") " pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:35.601469 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.601441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnpml\" (UniqueName: \"kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.601589 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.601475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.603778 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.603752 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.610563 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.610544 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnpml\" (UniqueName: \"kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml\") pod \"authorino-b58f66f6c-d9dsc\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.799898 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.799869 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:12:35.959731 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:35.959690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:12:35.962001 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:12:35.961965 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8cd3283_ac82_4f3a_a87d_a5be6557ab1e.slice/crio-4fd2aa8d70c2abde586b19daa35d027d64c84c1e9770e72690fda14d84a97738 WatchSource:0}: Error finding container 4fd2aa8d70c2abde586b19daa35d027d64c84c1e9770e72690fda14d84a97738: Status 404 returned error can't find the container with id 4fd2aa8d70c2abde586b19daa35d027d64c84c1e9770e72690fda14d84a97738 Jul 08 19:12:36.310950 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.310909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" event={"ID":"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e","Type":"ContainerStarted","Data":"4fd2aa8d70c2abde586b19daa35d027d64c84c1e9770e72690fda14d84a97738"} Jul 08 19:12:36.312426 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.312396 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" event={"ID":"8ceeb7ab-9c1b-48dd-b326-acf896770a6f","Type":"ContainerStarted","Data":"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98"} Jul 08 19:12:36.312567 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.312436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" event={"ID":"8ceeb7ab-9c1b-48dd-b326-acf896770a6f","Type":"ContainerStarted","Data":"f045c0423397266fc9cf4291df6dc1d605d4dcf02a54025d9d09079b87249d12"} Jul 08 19:12:36.312567 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.312456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:36.312567 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.312481 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" podUID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" containerName="authorino" containerID="cri-o://282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98" gracePeriod=30 Jul 08 19:12:36.327914 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.327871 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" podStartSLOduration=0.990985308 podStartE2EDuration="1.327855466s" totalImagesPullingTime="336.870158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:36.327409074 +0000 UTC m=+812.755297486" watchObservedRunningTime="2026-07-08 19:12:36.327855466 +0000 UTC m=+812.755743774" Jul 08 19:12:36.380865 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.380842 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:36.409209 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.409187 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74gsf\" (UniqueName: \"kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf\") pod \"957e1b1d-8e2f-4a92-aec9-a7911122ae71\" (UID: \"957e1b1d-8e2f-4a92-aec9-a7911122ae71\") " Jul 08 19:12:36.411058 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.411035 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf" (OuterVolumeSpecName: "kube-api-access-74gsf") pod "957e1b1d-8e2f-4a92-aec9-a7911122ae71" (UID: "957e1b1d-8e2f-4a92-aec9-a7911122ae71"). InnerVolumeSpecName "kube-api-access-74gsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:36.510031 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.509985 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74gsf\" (UniqueName: \"kubernetes.io/projected/957e1b1d-8e2f-4a92-aec9-a7911122ae71-kube-api-access-74gsf\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:12:36.577567 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.577540 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:36.610546 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.610521 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-999ff\" (UniqueName: \"kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff\") pod \"8ceeb7ab-9c1b-48dd-b326-acf896770a6f\" (UID: \"8ceeb7ab-9c1b-48dd-b326-acf896770a6f\") " Jul 08 19:12:36.612457 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.612426 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff" (OuterVolumeSpecName: "kube-api-access-999ff") pod "8ceeb7ab-9c1b-48dd-b326-acf896770a6f" (UID: "8ceeb7ab-9c1b-48dd-b326-acf896770a6f"). InnerVolumeSpecName "kube-api-access-999ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:36.711393 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:36.711360 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-999ff\" (UniqueName: \"kubernetes.io/projected/8ceeb7ab-9c1b-48dd-b326-acf896770a6f-kube-api-access-999ff\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:12:37.317701 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.317667 2580 generic.go:356] "Generic (PLEG): container finished" podID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" containerID="282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98" exitCode=0 Jul 08 19:12:37.318112 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.317711 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" Jul 08 19:12:37.318112 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.317752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" event={"ID":"8ceeb7ab-9c1b-48dd-b326-acf896770a6f","Type":"ContainerDied","Data":"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98"} Jul 08 19:12:37.318112 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.317790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rxjl9" event={"ID":"8ceeb7ab-9c1b-48dd-b326-acf896770a6f","Type":"ContainerDied","Data":"f045c0423397266fc9cf4291df6dc1d605d4dcf02a54025d9d09079b87249d12"} Jul 08 19:12:37.318112 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.317812 2580 scope.go:117] "RemoveContainer" containerID="282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98" Jul 08 19:12:37.319370 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.319349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b54745cf5-vtx5h" Jul 08 19:12:37.319469 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.319345 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" event={"ID":"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e","Type":"ContainerStarted","Data":"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056"} Jul 08 19:12:37.327325 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.327286 2580 scope.go:117] "RemoveContainer" containerID="282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98" Jul 08 19:12:37.327641 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:12:37.327614 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98\": container with ID starting with 282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98 not found: ID does not exist" containerID="282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98" Jul 08 19:12:37.327712 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.327653 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98"} err="failed to get container status \"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98\": rpc error: code = NotFound desc = could not find container \"282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98\": container with ID starting with 282b4a4145d63f91aa8fc01ea9e98613e055b0ef51a8159077c08747ee309b98 not found: ID does not exist" Jul 08 19:12:37.338987 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.338945 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" podStartSLOduration=1.95960394 podStartE2EDuration="2.338931489s" totalImagesPullingTime="379.327549ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:37.338041313 +0000 UTC m=+813.765929621" watchObservedRunningTime="2026-07-08 19:12:37.338931489 +0000 UTC m=+813.766819796" Jul 08 19:12:37.356548 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.356519 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:37.361422 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.361395 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rxjl9"] Jul 08 19:12:37.390708 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.390676 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b54745cf5-vtx5h"] Jul 08 19:12:37.396089 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:37.396041 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5b54745cf5-vtx5h"] Jul 08 19:12:38.213729 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:38.213697 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" path="/var/lib/kubelet/pods/8ceeb7ab-9c1b-48dd-b326-acf896770a6f/volumes" Jul 08 19:12:38.214024 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:12:38.214012 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="957e1b1d-8e2f-4a92-aec9-a7911122ae71" path="/var/lib/kubelet/pods/957e1b1d-8e2f-4a92-aec9-a7911122ae71/volumes" Jul 08 19:13:16.796504 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:13:16.796477 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:13:34.311209 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:13:34.311163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:13:46.006883 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:13:46.006840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:14:00.013439 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:00.013355 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:14:04.074457 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:04.074429 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:14:04.077352 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:04.077331 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:14:09.814560 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:09.814523 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:14:29.904376 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:29.904337 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:14:43.491881 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.491845 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt"] Jul 08 19:14:43.492264 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.492177 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" containerName="authorino" Jul 08 19:14:43.492264 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.492187 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" containerName="authorino" Jul 08 19:14:43.492264 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.492251 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ceeb7ab-9c1b-48dd-b326-acf896770a6f" containerName="authorino" Jul 08 19:14:43.495103 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.495088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.497803 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.497748 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:14:43.497803 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.497748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hs4tk\"" Jul 08 19:14:43.497986 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.497835 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:14:43.498961 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.498938 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 19:14:43.503848 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.503827 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt"] Jul 08 19:14:43.528476 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.528583 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.528583 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.528583 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.528693 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvqkb\" (UniqueName: \"kubernetes.io/projected/04d87c5a-e379-41f1-b7a9-d387e02c1c97-kube-api-access-kvqkb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.528693 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.528647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629273 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629428 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629428 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629318 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629516 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvqkb\" (UniqueName: \"kubernetes.io/projected/04d87c5a-e379-41f1-b7a9-d387e02c1c97-kube-api-access-kvqkb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629516 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629615 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629666 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629727 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629709 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.629832 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.629815 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.631482 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.631466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04d87c5a-e379-41f1-b7a9-d387e02c1c97-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.631869 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.631849 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04d87c5a-e379-41f1-b7a9-d387e02c1c97-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.638630 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.638604 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvqkb\" (UniqueName: \"kubernetes.io/projected/04d87c5a-e379-41f1-b7a9-d387e02c1c97-kube-api-access-kvqkb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt\" (UID: \"04d87c5a-e379-41f1-b7a9-d387e02c1c97\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.805137 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.805071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:43.939994 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.939971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt"] Jul 08 19:14:43.942565 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:14:43.942531 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04d87c5a_e379_41f1_b7a9_d387e02c1c97.slice/crio-289c812482735caf33b727b42d11d04b9fa99d86a0fafe78580a793ce97d785f WatchSource:0}: Error finding container 289c812482735caf33b727b42d11d04b9fa99d86a0fafe78580a793ce97d785f: Status 404 returned error can't find the container with id 289c812482735caf33b727b42d11d04b9fa99d86a0fafe78580a793ce97d785f Jul 08 19:14:43.943819 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:43.943805 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:14:44.877931 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:44.877885 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" event={"ID":"04d87c5a-e379-41f1-b7a9-d387e02c1c97","Type":"ContainerStarted","Data":"289c812482735caf33b727b42d11d04b9fa99d86a0fafe78580a793ce97d785f"} Jul 08 19:14:45.884021 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:45.883986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" event={"ID":"04d87c5a-e379-41f1-b7a9-d387e02c1c97","Type":"ContainerStarted","Data":"d0b3deeda2cf2a78c8f4ea394c13dffd1f672100926ac89824d6dbcc53c5eed0"} Jul 08 19:14:45.902790 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:45.902739 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" podStartSLOduration=1.811033395 podStartE2EDuration="2.90272216s" totalImagesPullingTime="1.091688765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:45.90159089 +0000 UTC m=+942.329479198" watchObservedRunningTime="2026-07-08 19:14:45.90272216 +0000 UTC m=+942.330610468" Jul 08 19:14:46.394825 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:46.394781 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:14:53.805646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:53.805600 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:53.805646 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:53.805655 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:53.818246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:53.818224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:14:53.934568 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:14:53.934540 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt" Jul 08 19:15:24.491777 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.491741 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-664859bd6-cxlzz"] Jul 08 19:15:24.497818 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.497794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.506406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.506376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664859bd6-cxlzz"] Jul 08 19:15:24.665901 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.665868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/def91852-0d09-4db2-9e14-99f44b239c06-tls-cert\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.666089 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.665917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddsdq\" (UniqueName: \"kubernetes.io/projected/def91852-0d09-4db2-9e14-99f44b239c06-kube-api-access-ddsdq\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.767245 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.767152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/def91852-0d09-4db2-9e14-99f44b239c06-tls-cert\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.767245 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.767200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddsdq\" (UniqueName: \"kubernetes.io/projected/def91852-0d09-4db2-9e14-99f44b239c06-kube-api-access-ddsdq\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.769718 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.769694 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/def91852-0d09-4db2-9e14-99f44b239c06-tls-cert\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.775458 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.775437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddsdq\" (UniqueName: \"kubernetes.io/projected/def91852-0d09-4db2-9e14-99f44b239c06-kube-api-access-ddsdq\") pod \"authorino-664859bd6-cxlzz\" (UID: \"def91852-0d09-4db2-9e14-99f44b239c06\") " pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.807394 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.807369 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664859bd6-cxlzz" Jul 08 19:15:24.950654 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:24.950625 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664859bd6-cxlzz"] Jul 08 19:15:24.952374 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:15:24.952345 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddef91852_0d09_4db2_9e14_99f44b239c06.slice/crio-4f7f425ed25740a64b53a246e443f4e97616a528c914ba3e6e9cbe5aeb44cfd7 WatchSource:0}: Error finding container 4f7f425ed25740a64b53a246e443f4e97616a528c914ba3e6e9cbe5aeb44cfd7: Status 404 returned error can't find the container with id 4f7f425ed25740a64b53a246e443f4e97616a528c914ba3e6e9cbe5aeb44cfd7 Jul 08 19:15:25.059834 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:25.059759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664859bd6-cxlzz" event={"ID":"def91852-0d09-4db2-9e14-99f44b239c06","Type":"ContainerStarted","Data":"4f7f425ed25740a64b53a246e443f4e97616a528c914ba3e6e9cbe5aeb44cfd7"} Jul 08 19:15:26.068256 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.068215 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664859bd6-cxlzz" event={"ID":"def91852-0d09-4db2-9e14-99f44b239c06","Type":"ContainerStarted","Data":"581e396ea5e811ce90b3af349190a6432052db39dbfc3461966e4f00c3460a59"} Jul 08 19:15:26.100140 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.100074 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-664859bd6-cxlzz" podStartSLOduration=1.678932454 podStartE2EDuration="2.100054349s" totalImagesPullingTime="421.121895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:26.09743476 +0000 UTC m=+982.525323071" watchObservedRunningTime="2026-07-08 19:15:26.100054349 +0000 UTC m=+982.527942659" Jul 08 19:15:26.162268 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.162229 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:15:26.162683 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.162648 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" podUID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" containerName="authorino" containerID="cri-o://f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056" gracePeriod=30 Jul 08 19:15:26.438199 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.438171 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:15:26.591565 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.591479 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert\") pod \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " Jul 08 19:15:26.591565 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.591528 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnpml\" (UniqueName: \"kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml\") pod \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\" (UID: \"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e\") " Jul 08 19:15:26.593802 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.593759 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml" (OuterVolumeSpecName: "kube-api-access-jnpml") pod "d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" (UID: "d8cd3283-ac82-4f3a-a87d-a5be6557ab1e"). InnerVolumeSpecName "kube-api-access-jnpml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:15:26.602386 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.602355 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" (UID: "d8cd3283-ac82-4f3a-a87d-a5be6557ab1e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:15:26.692859 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.692821 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-tls-cert\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:15:26.692859 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:26.692854 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnpml\" (UniqueName: \"kubernetes.io/projected/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e-kube-api-access-jnpml\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:15:27.075938 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.075898 2580 generic.go:356] "Generic (PLEG): container finished" podID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" containerID="f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056" exitCode=0 Jul 08 19:15:27.076368 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.075948 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" Jul 08 19:15:27.076368 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.075948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" event={"ID":"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e","Type":"ContainerDied","Data":"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056"} Jul 08 19:15:27.076368 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.076002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b58f66f6c-d9dsc" event={"ID":"d8cd3283-ac82-4f3a-a87d-a5be6557ab1e","Type":"ContainerDied","Data":"4fd2aa8d70c2abde586b19daa35d027d64c84c1e9770e72690fda14d84a97738"} Jul 08 19:15:27.076368 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.076027 2580 scope.go:117] "RemoveContainer" containerID="f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056" Jul 08 19:15:27.084973 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.084955 2580 scope.go:117] "RemoveContainer" containerID="f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056" Jul 08 19:15:27.085248 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:15:27.085227 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056\": container with ID starting with f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056 not found: ID does not exist" containerID="f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056" Jul 08 19:15:27.085350 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.085260 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056"} err="failed to get container status \"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056\": rpc error: code = NotFound desc = could not find container \"f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056\": container with ID starting with f112da0cc35f965816f5e907f51bf9b2574a1b1db005669aeff1bf2925c57056 not found: ID does not exist" Jul 08 19:15:27.097063 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.097040 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:15:27.103295 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:27.103262 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b58f66f6c-d9dsc"] Jul 08 19:15:28.213518 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:28.213487 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" path="/var/lib/kubelet/pods/d8cd3283-ac82-4f3a-a87d-a5be6557ab1e/volumes" Jul 08 19:15:54.407649 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:15:54.407612 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:16:03.696640 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:16:03.696598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:16:15.096597 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:16:15.096558 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:16:22.901850 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:16:22.901797 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:16:34.692510 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:16:34.692472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:17:02.496020 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:17:02.495943 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:17:16.600136 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:17:16.600100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:17:50.996202 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:17:50.996162 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:17:58.298414 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:17:58.298376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:07.501984 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:07.501945 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:14.909573 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:14.909537 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:25.196290 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:25.196257 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:34.597335 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:34.597237 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:45.826708 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:45.826671 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:18:55.928470 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:18:55.928425 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:19:04.102991 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:19:04.102958 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:19:04.106911 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:19:04.106888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:19:18.424642 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:19:18.424601 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:19:27.705714 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:19:27.705680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:19:51.502703 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:19:51.502668 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:02.410118 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:02.410031 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:17.500975 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:17.500939 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:26.908418 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:26.908380 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:38.015831 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:38.015794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:47.497277 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:47.497238 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:20:56.510909 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:20:56.510875 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:22:43.315722 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:22:43.315687 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:22:54.911157 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:22:54.911117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:23:04.024228 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:23:04.024151 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:23:16.347253 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:23:16.347218 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:23:31.347098 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:23:31.347061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:23:43.428421 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:23:43.428386 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:23:53.309249 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:23:53.309211 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:24:03.240282 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:24:03.240245 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:24:04.129098 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:24:04.129070 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:24:04.134709 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:24:04.134687 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:24:13.463512 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:24:13.463475 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:24:51.133467 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:24:51.133408 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:01.454958 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:01.454921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:11.631801 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:11.631767 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:23.914806 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:23.914773 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:39.528258 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:39.528219 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:49.003902 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:49.003861 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:25:58.263975 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:25:58.263892 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:26:07.506122 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:26:07.506081 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:26:18.605018 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:26:18.604982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:26:28.412161 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:26:28.412130 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:26:41.508364 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:26:41.508326 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:26:50.609374 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:26:50.609337 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:00.298959 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:00.298918 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:09.839326 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:09.838945 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:23.003244 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:23.003206 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:31.917430 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:31.917343 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:43.303532 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:43.303498 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:27:55.106403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:27:55.106369 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:08.002553 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:08.002514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:17.115378 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:17.115338 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:29.709986 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:29.709953 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:38.706692 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:38.706645 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:49.312866 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:49.312833 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:28:58.305783 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:28:58.305750 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:29:04.155905 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:29:04.155862 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:29:04.163068 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:29:04.163050 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:29:21.230531 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:29:21.230495 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:29:30.217319 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:29:30.217272 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:30:54.419387 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:30:54.419354 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:31:02.520484 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:31:02.520446 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:31:10.116922 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:31:10.116872 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:31:20.315101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:31:20.315067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:31:31.913268 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:31:31.913235 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:32:05.804934 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:05.804854 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:32:09.812429 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.812394 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:09.812786 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.812709 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" containerName="authorino" Jul 08 19:32:09.812786 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.812720 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" containerName="authorino" Jul 08 19:32:09.812786 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.812786 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8cd3283-ac82-4f3a-a87d-a5be6557ab1e" containerName="authorino" Jul 08 19:32:09.815578 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.815560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.820020 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.819997 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-90938a12\"/\"default-dockercfg-jw6jx\"" Jul 08 19:32:09.820135 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.820002 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-90938a12\"/\"openshift-service-ca.crt\"" Jul 08 19:32:09.820135 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.820002 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-90938a12\"/\"e2e-lifecycle-model-90938a12-kserve-self-signed-certs\"" Jul 08 19:32:09.820744 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.820729 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-90938a12\"/\"kube-root-ca.crt\"" Jul 08 19:32:09.825543 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.825520 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:09.869812 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.869951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.869951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.869951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.869951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrpz\" (UniqueName: \"kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.869951 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.869933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.970933 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.970897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.970933 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.970938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971166 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.970959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971166 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.970981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971166 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.971014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrpz\" (UniqueName: \"kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971166 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.971065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971431 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.971405 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971497 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.971478 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.971575 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.971551 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.973335 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.973290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.973481 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.973467 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:09.980935 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:09.980909 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrpz\" (UniqueName: \"kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz\") pod \"e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:10.125482 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:10.125363 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:10.260421 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:10.260395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:10.263140 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:32:10.263110 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06d30fd3_ddb8_4334_8704_a758d3f79ec8.slice/crio-d8b29c41f55a590155ce064dfb0a6a9a19e573003694104bc657541583a5ecf4 WatchSource:0}: Error finding container d8b29c41f55a590155ce064dfb0a6a9a19e573003694104bc657541583a5ecf4: Status 404 returned error can't find the container with id d8b29c41f55a590155ce064dfb0a6a9a19e573003694104bc657541583a5ecf4 Jul 08 19:32:10.511833 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:10.511795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" event={"ID":"06d30fd3-ddb8-4334-8704-a758d3f79ec8","Type":"ContainerStarted","Data":"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7"} Jul 08 19:32:10.511833 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:10.511841 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" event={"ID":"06d30fd3-ddb8-4334-8704-a758d3f79ec8","Type":"ContainerStarted","Data":"d8b29c41f55a590155ce064dfb0a6a9a19e573003694104bc657541583a5ecf4"} Jul 08 19:32:10.535527 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:10.535459 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" podStartSLOduration=1.535444498 podStartE2EDuration="1.535444498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:10.533929738 +0000 UTC m=+1986.961818041" watchObservedRunningTime="2026-07-08 19:32:10.535444498 +0000 UTC m=+1986.963332806" Jul 08 19:32:15.103192 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:15.103155 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:32:20.126249 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:20.126150 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:20.126249 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:20.126206 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:20.138597 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:20.138571 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:20.565576 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:20.565539 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:28.104356 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:28.104319 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:32:41.411745 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:41.411710 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:32:41.688340 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:41.688235 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:56.708539 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:56.708499 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" podUID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" containerName="main" containerID="cri-o://71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7" gracePeriod=45 Jul 08 19:32:56.962026 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:56.961971 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:57.056430 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056404 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056588 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056446 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056588 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056489 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056588 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056508 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsrpz\" (UniqueName: \"kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056588 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056539 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056810 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056601 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir\") pod \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\" (UID: \"06d30fd3-ddb8-4334-8704-a758d3f79ec8\") " Jul 08 19:32:57.056810 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056771 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache" (OuterVolumeSpecName: "model-cache") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:57.056917 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056867 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-model-cache\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.056917 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056868 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home" (OuterVolumeSpecName: "home") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:57.056993 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.056918 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:57.058612 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.058590 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:32:57.058612 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.058605 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm" (OuterVolumeSpecName: "dshm") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:57.058726 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.058620 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz" (OuterVolumeSpecName: "kube-api-access-lsrpz") pod "06d30fd3-ddb8-4334-8704-a758d3f79ec8" (UID: "06d30fd3-ddb8-4334-8704-a758d3f79ec8"). InnerVolumeSpecName "kube-api-access-lsrpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:57.157678 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.157648 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tmp-dir\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.157678 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.157672 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-dshm\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.157678 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.157681 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d30fd3-ddb8-4334-8704-a758d3f79ec8-tls-certs\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.157847 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.157690 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsrpz\" (UniqueName: \"kubernetes.io/projected/06d30fd3-ddb8-4334-8704-a758d3f79ec8-kube-api-access-lsrpz\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.157847 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.157699 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d30fd3-ddb8-4334-8704-a758d3f79ec8-home\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:32:57.721753 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.721721 2580 generic.go:356] "Generic (PLEG): container finished" podID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" containerID="71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7" exitCode=0 Jul 08 19:32:57.722293 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.721762 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" Jul 08 19:32:57.722293 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.721809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" event={"ID":"06d30fd3-ddb8-4334-8704-a758d3f79ec8","Type":"ContainerDied","Data":"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7"} Jul 08 19:32:57.722293 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.721856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk" event={"ID":"06d30fd3-ddb8-4334-8704-a758d3f79ec8","Type":"ContainerDied","Data":"d8b29c41f55a590155ce064dfb0a6a9a19e573003694104bc657541583a5ecf4"} Jul 08 19:32:57.722293 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.721878 2580 scope.go:117] "RemoveContainer" containerID="71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7" Jul 08 19:32:57.731000 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.730985 2580 scope.go:117] "RemoveContainer" containerID="71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7" Jul 08 19:32:57.731263 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:32:57.731246 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7\": container with ID starting with 71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7 not found: ID does not exist" containerID="71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7" Jul 08 19:32:57.731345 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.731273 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7"} err="failed to get container status \"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7\": rpc error: code = NotFound desc = could not find container \"71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7\": container with ID starting with 71d2613d4aa45e7cbb25e172915eaaff7713d1d89d911d3cced79a21989086b7 not found: ID does not exist" Jul 08 19:32:57.742713 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.742690 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:57.746896 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:57.746876 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-90938a12/e2e-lifecycle-model-90938a12-kserve-75ff955f99-6t6rk"] Jul 08 19:32:58.214820 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:32:58.214792 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" path="/var/lib/kubelet/pods/06d30fd3-ddb8-4334-8704-a758d3f79ec8/volumes" Jul 08 19:33:21.005430 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:33:21.005393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:33:31.206564 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:33:31.206483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:33:43.403701 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:33:43.403660 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:34:04.182214 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:04.182185 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:34:04.189803 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:04.189781 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:34:32.528944 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:32.528913 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:34:43.697234 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.697199 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:34:43.994790 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.994761 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:34:43.995146 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.995131 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" containerName="main" Jul 08 19:34:43.995246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.995150 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" containerName="main" Jul 08 19:34:43.995325 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.995255 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d30fd3-ddb8-4334-8704-a758d3f79ec8" containerName="main" Jul 08 19:34:43.998158 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:43.998139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.000855 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.000834 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5fb4a8\"/\"openshift-service-ca.crt\"" Jul 08 19:34:44.001824 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.001800 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-5fb4a8\"/\"kube-root-ca.crt\"" Jul 08 19:34:44.001916 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.001864 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5fb4a8\"/\"test-model-5fb4a8-kserve-self-signed-certs\"" Jul 08 19:34:44.001916 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.001865 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-5fb4a8\"/\"default-dockercfg-5szjp\"" Jul 08 19:34:44.008200 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.008178 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:34:44.133280 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.133411 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.133411 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.133411 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbr75\" (UniqueName: \"kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.133517 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.133517 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.133487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234266 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234416 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234416 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234416 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbr75\" (UniqueName: \"kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234416 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234606 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234667 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234636 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234826 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234781 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.234892 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.234811 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.236623 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.236603 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.236859 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.236842 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.242278 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.242260 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbr75\" (UniqueName: \"kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75\") pod \"test-model-5fb4a8-kserve-cb945d98f-khqbb\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.307617 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.307558 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:44.435826 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:44.435798 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:34:44.438041 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:34:44.438016 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bfba3ed_3ba8_4c30_a840_5125af2a1ff7.slice/crio-aed74c0281bcdf55836330d7bafca2492ee45b9bbe8d62a0f66270888c13c77a WatchSource:0}: Error finding container aed74c0281bcdf55836330d7bafca2492ee45b9bbe8d62a0f66270888c13c77a: Status 404 returned error can't find the container with id aed74c0281bcdf55836330d7bafca2492ee45b9bbe8d62a0f66270888c13c77a Jul 08 19:34:45.211197 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:45.211163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" event={"ID":"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7","Type":"ContainerStarted","Data":"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0"} Jul 08 19:34:45.211197 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:45.211200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" event={"ID":"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7","Type":"ContainerStarted","Data":"aed74c0281bcdf55836330d7bafca2492ee45b9bbe8d62a0f66270888c13c77a"} Jul 08 19:34:45.229276 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:45.229227 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" podStartSLOduration=2.229212568 podStartE2EDuration="2.229212568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:45.228848345 +0000 UTC m=+2141.656736652" watchObservedRunningTime="2026-07-08 19:34:45.229212568 +0000 UTC m=+2141.657100879" Jul 08 19:34:53.899150 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:53.899113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:34:54.307899 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:54.307861 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:54.308064 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:54.307911 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:54.320190 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:54.320161 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:34:55.264651 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:34:55.264623 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:35:22.701393 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:22.701358 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:35:35.443463 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:35.443432 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:35:40.795116 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:40.795079 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:35:50.463499 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.463449 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" podUID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" containerName="main" containerID="cri-o://c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0" gracePeriod=45 Jul 08 19:35:50.720928 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.720871 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:35:50.779195 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779166 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbr75\" (UniqueName: \"kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779195 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779196 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779471 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779233 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779471 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779279 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779471 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779348 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779471 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779407 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home\") pod \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\" (UID: \"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7\") " Jul 08 19:35:50.779672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779502 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:35:50.779672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779644 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home" (OuterVolumeSpecName: "home") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:35:50.779672 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779663 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-model-cache\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.779785 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.779650 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:35:50.781406 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.781384 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75" (OuterVolumeSpecName: "kube-api-access-lbr75") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "kube-api-access-lbr75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:35:50.781730 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.781710 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm" (OuterVolumeSpecName: "dshm") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:35:50.781730 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.781722 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" (UID: "7bfba3ed-3ba8-4c30-a840-5125af2a1ff7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:35:50.880403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.880380 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-dshm\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.880403 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.880402 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tmp-dir\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.880596 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.880413 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-tls-certs\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.880596 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.880421 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-home\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.880596 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:50.880429 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbr75\" (UniqueName: \"kubernetes.io/projected/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7-kube-api-access-lbr75\") on node \"ip-10-0-142-223.ec2.internal\" DevicePath \"\"" Jul 08 19:35:51.511626 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.511590 2580 generic.go:356] "Generic (PLEG): container finished" podID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" containerID="c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0" exitCode=0 Jul 08 19:35:51.512101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.511635 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" Jul 08 19:35:51.512101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.511661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" event={"ID":"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7","Type":"ContainerDied","Data":"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0"} Jul 08 19:35:51.512101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.511696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb" event={"ID":"7bfba3ed-3ba8-4c30-a840-5125af2a1ff7","Type":"ContainerDied","Data":"aed74c0281bcdf55836330d7bafca2492ee45b9bbe8d62a0f66270888c13c77a"} Jul 08 19:35:51.512101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.511714 2580 scope.go:117] "RemoveContainer" containerID="c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0" Jul 08 19:35:51.521382 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.521363 2580 scope.go:117] "RemoveContainer" containerID="c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0" Jul 08 19:35:51.521656 ip-10-0-142-223 kubenswrapper[2580]: E0708 19:35:51.521635 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0\": container with ID starting with c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0 not found: ID does not exist" containerID="c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0" Jul 08 19:35:51.521702 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.521665 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0"} err="failed to get container status \"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0\": rpc error: code = NotFound desc = could not find container \"c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0\": container with ID starting with c381fe52c3a598a211eb17820403350a9fbbfa118b4dda6d720c7cc78afaf9f0 not found: ID does not exist" Jul 08 19:35:51.534736 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.534712 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:35:51.538709 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:51.538685 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-5fb4a8/test-model-5fb4a8-kserve-cb945d98f-khqbb"] Jul 08 19:35:52.214444 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:35:52.214411 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" path="/var/lib/kubelet/pods/7bfba3ed-3ba8-4c30-a840-5125af2a1ff7/volumes" Jul 08 19:36:23.493316 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:36:23.493271 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:36:38.921157 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:36:38.921075 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:36:55.895886 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:36:55.895845 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:37:09.097905 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:37:09.097868 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:37:20.993502 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:37:20.993467 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:39:04.208971 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:39:04.208939 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:39:04.217275 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:39:04.217256 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cgfgt_83c21736-acef-47c9-832d-f729ece41b4d/ovn-acl-logging/0.log" Jul 08 19:40:30.893978 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:30.893945 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:40:41.840049 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:41.840014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664859bd6-cxlzz_def91852-0d09-4db2-9e14-99f44b239c06/authorino/0.log" Jul 08 19:40:44.000233 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:44.000195 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4ctp"] Jul 08 19:40:46.954257 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:46.954220 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pffl8_d3d98d42-4048-4c2a-9e6b-2c10b980d744/manager/0.log" Jul 08 19:40:47.199432 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:47.199401 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd7775959-dbrvn_df3f5e05-9c06-4511-90c2-21dd0dd58469/copy-manifests/0.log" Jul 08 19:40:47.217612 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:47.217583 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd7775959-dbrvn_df3f5e05-9c06-4511-90c2-21dd0dd58469/manager/0.log" Jul 08 19:40:48.647017 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:48.646984 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664859bd6-cxlzz_def91852-0d09-4db2-9e14-99f44b239c06/authorino/0.log" Jul 08 19:40:48.995365 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:48.995337 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9zjgl_5de39805-9ccd-439b-beff-4361c91988cf/kuadrant-console-plugin/0.log" Jul 08 19:40:49.355007 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:49.354927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b4ctp_a1d0bcb7-f160-441d-9c83-67d8de69ad9a/limitador/0.log" Jul 08 19:40:50.178155 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:50.178129 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85bd9454df-q9k5c_f107e6dd-9b2b-42a4-b186-04b4b2f778a5/kube-auth-proxy/0.log" Jul 08 19:40:50.642223 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:50.642193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cf956c5dc-r496w_125a062b-4f51-4040-bb38-f0a8a610d369/router/0.log" Jul 08 19:40:51.585500 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:51.585473 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-cpgzt_04d87c5a-e379-41f1-b7a9-d387e02c1c97/main/0.log" Jul 08 19:40:56.092634 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.092556 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c7j27/must-gather-js7v5"] Jul 08 19:40:56.092983 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.092876 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" containerName="main" Jul 08 19:40:56.092983 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.092886 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" containerName="main" Jul 08 19:40:56.092983 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.092977 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfba3ed-3ba8-4c30-a840-5125af2a1ff7" containerName="main" Jul 08 19:40:56.096192 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.096172 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.099064 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.099046 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7j27\"/\"kube-root-ca.crt\"" Jul 08 19:40:56.099974 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.099957 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c7j27\"/\"default-dockercfg-z67fq\"" Jul 08 19:40:56.100070 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.099990 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7j27\"/\"openshift-service-ca.crt\"" Jul 08 19:40:56.104170 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.104147 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7j27/must-gather-js7v5"] Jul 08 19:40:56.172615 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.172587 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9ecf59a-45ce-4860-a320-5871f1095897-must-gather-output\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.172752 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.172627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wt59\" (UniqueName: \"kubernetes.io/projected/f9ecf59a-45ce-4860-a320-5871f1095897-kube-api-access-7wt59\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.273743 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.273714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wt59\" (UniqueName: \"kubernetes.io/projected/f9ecf59a-45ce-4860-a320-5871f1095897-kube-api-access-7wt59\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.273890 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.273868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9ecf59a-45ce-4860-a320-5871f1095897-must-gather-output\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.274171 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.274156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9ecf59a-45ce-4860-a320-5871f1095897-must-gather-output\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.281519 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.281494 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wt59\" (UniqueName: \"kubernetes.io/projected/f9ecf59a-45ce-4860-a320-5871f1095897-kube-api-access-7wt59\") pod \"must-gather-js7v5\" (UID: \"f9ecf59a-45ce-4860-a320-5871f1095897\") " pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.406502 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.406433 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7j27/must-gather-js7v5" Jul 08 19:40:56.532251 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.532226 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7j27/must-gather-js7v5"] Jul 08 19:40:56.534732 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:40:56.534702 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9ecf59a_45ce_4860_a320_5871f1095897.slice/crio-3b7ba5b2e16f26e2afc36b7707c45569c04509939e2ad1729cf3a4c3bcd3b151 WatchSource:0}: Error finding container 3b7ba5b2e16f26e2afc36b7707c45569c04509939e2ad1729cf3a4c3bcd3b151: Status 404 returned error can't find the container with id 3b7ba5b2e16f26e2afc36b7707c45569c04509939e2ad1729cf3a4c3bcd3b151 Jul 08 19:40:56.536589 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.536573 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:40:56.882262 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:56.882224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7j27/must-gather-js7v5" event={"ID":"f9ecf59a-45ce-4860-a320-5871f1095897","Type":"ContainerStarted","Data":"3b7ba5b2e16f26e2afc36b7707c45569c04509939e2ad1729cf3a4c3bcd3b151"} Jul 08 19:40:57.895246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:57.895153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7j27/must-gather-js7v5" event={"ID":"f9ecf59a-45ce-4860-a320-5871f1095897","Type":"ContainerStarted","Data":"11dc4f92709253274ffc0b814086f498e6f970287fe31072622e0dad099532e4"} Jul 08 19:40:57.895246 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:57.895210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7j27/must-gather-js7v5" event={"ID":"f9ecf59a-45ce-4860-a320-5871f1095897","Type":"ContainerStarted","Data":"6bb575c9ad4822b4612512bff4aace46364f75eb97fd9402246cf20b2e6f633f"} Jul 08 19:40:57.912277 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:57.912219 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c7j27/must-gather-js7v5" podStartSLOduration=0.973366589 podStartE2EDuration="1.912198743s" totalImagesPullingTime="938.832154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:40:57.912177594 +0000 UTC m=+2514.340065903" watchObservedRunningTime="2026-07-08 19:40:57.912198743 +0000 UTC m=+2514.340087052" Jul 08 19:40:59.166249 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:59.166218 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hclm2_ffea4bdc-91db-45ba-9dee-f71d26b67dbd/global-pull-secret-syncer/0.log" Jul 08 19:40:59.292169 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:59.292134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lvtvj_61016499-c57a-4ff7-8131-62c1ef94de54/konnectivity-agent/0.log" Jul 08 19:40:59.402255 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:40:59.402215 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-223.ec2.internal_872c3c853569ccf8888d4463b966dc40/haproxy/0.log" Jul 08 19:41:04.163757 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:04.163722 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664859bd6-cxlzz_def91852-0d09-4db2-9e14-99f44b239c06/authorino/0.log" Jul 08 19:41:04.272496 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:04.272462 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9zjgl_5de39805-9ccd-439b-beff-4361c91988cf/kuadrant-console-plugin/0.log" Jul 08 19:41:04.454678 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:04.454597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b4ctp_a1d0bcb7-f160-441d-9c83-67d8de69ad9a/limitador/0.log" Jul 08 19:41:06.319681 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:06.319641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-fjb2n_a424b5c6-1fc5-44fd-bc0f-7572c9db3b7b/cluster-monitoring-operator/0.log" Jul 08 19:41:06.568562 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:06.568535 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-gm4vn_7c044123-79f8-412e-93a5-26afda868b32/monitoring-plugin/0.log" Jul 08 19:41:06.611123 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:06.611052 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7t66q_e3a25cd9-e504-4330-9245-a9c0d1f4218b/node-exporter/0.log" Jul 08 19:41:06.638770 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:06.638736 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7t66q_e3a25cd9-e504-4330-9245-a9c0d1f4218b/kube-rbac-proxy/0.log" Jul 08 19:41:06.662680 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:06.662649 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7t66q_e3a25cd9-e504-4330-9245-a9c0d1f4218b/init-textfile/0.log" Jul 08 19:41:07.143232 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.143190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q4sdz_38aba6cc-6f1b-40e9-bab3-81c0c95babe2/prometheus-operator/0.log" Jul 08 19:41:07.162105 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.162080 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-q4sdz_38aba6cc-6f1b-40e9-bab3-81c0c95babe2/kube-rbac-proxy/0.log" Jul 08 19:41:07.301896 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.301850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/thanos-query/0.log" Jul 08 19:41:07.325844 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.325811 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/kube-rbac-proxy-web/0.log" Jul 08 19:41:07.346568 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.346538 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/kube-rbac-proxy/0.log" Jul 08 19:41:07.367101 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.367073 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/prom-label-proxy/0.log" Jul 08 19:41:07.389455 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.389424 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/kube-rbac-proxy-rules/0.log" Jul 08 19:41:07.413705 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:07.413624 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f4948666f-75pkn_2bf71693-6f61-4e64-a6d4-7296c19b656f/kube-rbac-proxy-metrics/0.log" Jul 08 19:41:09.340224 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.340185 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj"] Jul 08 19:41:09.352228 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.352190 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.361339 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.358888 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj"] Jul 08 19:41:09.521418 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.521364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-lib-modules\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.521610 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.521428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-podres\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.521610 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.521470 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-proc\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.521610 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.521528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-sys\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.521610 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.521576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmt5s\" (UniqueName: \"kubernetes.io/projected/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-kube-api-access-pmt5s\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623082 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.622979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-lib-modules\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623082 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-podres\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623355 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-proc\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623355 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623202 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-proc\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623355 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-sys\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623530 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmt5s\" (UniqueName: \"kubernetes.io/projected/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-kube-api-access-pmt5s\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623746 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623721 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-podres\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623871 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623803 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-sys\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.623962 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.623896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-lib-modules\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.632742 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.632716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmt5s\" (UniqueName: \"kubernetes.io/projected/58d86bb8-5e8c-4e48-ac72-0439e47e2fcb-kube-api-access-pmt5s\") pod \"perf-node-gather-daemonset-fvwbj\" (UID: \"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb\") " pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.668125 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.668097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:09.843684 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.842473 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj"] Jul 08 19:41:09.845970 ip-10-0-142-223 kubenswrapper[2580]: W0708 19:41:09.845837 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod58d86bb8_5e8c_4e48_ac72_0439e47e2fcb.slice/crio-037c46a49247a88e750f91a86697b8ccaa14314a844065c631158f3e579200e0 WatchSource:0}: Error finding container 037c46a49247a88e750f91a86697b8ccaa14314a844065c631158f3e579200e0: Status 404 returned error can't find the container with id 037c46a49247a88e750f91a86697b8ccaa14314a844065c631158f3e579200e0 Jul 08 19:41:09.996478 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.996441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" event={"ID":"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb","Type":"ContainerStarted","Data":"559ef00b2d8ac8ce7a6c69303c9fa4dd1c2a46bca40e512d3b6da5780d1bdf6b"} Jul 08 19:41:09.996647 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.996501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" event={"ID":"58d86bb8-5e8c-4e48-ac72-0439e47e2fcb","Type":"ContainerStarted","Data":"037c46a49247a88e750f91a86697b8ccaa14314a844065c631158f3e579200e0"} Jul 08 19:41:09.996647 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:09.996539 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:10.014350 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:10.014269 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" podStartSLOduration=1.01424776 podStartE2EDuration="1.01424776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:41:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:41:10.012352793 +0000 UTC m=+2526.440241101" watchObservedRunningTime="2026-07-08 19:41:10.01424776 +0000 UTC m=+2526.442136069" Jul 08 19:41:11.652648 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:11.652619 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdx2v_a5a50da0-90dd-407a-a1bf-b57bd756124e/dns/0.log" Jul 08 19:41:11.676043 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:11.676016 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vdx2v_a5a50da0-90dd-407a-a1bf-b57bd756124e/kube-rbac-proxy/0.log" Jul 08 19:41:11.730291 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:11.730258 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rtnkj_270c8cba-3f78-4663-85e1-ba8d92d58158/dns-node-resolver/0.log" Jul 08 19:41:12.325516 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:12.325487 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-655q2_58595100-3eee-4a74-9476-e81e8609049c/node-ca/0.log" Jul 08 19:41:13.414957 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:13.414929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85bd9454df-q9k5c_f107e6dd-9b2b-42a4-b186-04b4b2f778a5/kube-auth-proxy/0.log" Jul 08 19:41:13.574128 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:13.574082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cf956c5dc-r496w_125a062b-4f51-4040-bb38-f0a8a610d369/router/0.log" Jul 08 19:41:14.275612 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:14.275576 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n9mqf_b4a10718-e056-4a7a-851b-be327aff9606/serve-healthcheck-canary/0.log" Jul 08 19:41:14.770065 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:14.770035 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjtqk_c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea/kube-rbac-proxy/0.log" Jul 08 19:41:14.811895 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:14.811870 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjtqk_c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea/exporter/0.log" Jul 08 19:41:14.848660 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:14.848634 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjtqk_c7ea5ba4-7dae-477a-abfe-5e1e875cc4ea/extractor/0.log" Jul 08 19:41:16.012850 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:16.012813 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c7j27/perf-node-gather-daemonset-fvwbj" Jul 08 19:41:17.425586 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:17.425555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pffl8_d3d98d42-4048-4c2a-9e6b-2c10b980d744/manager/0.log" Jul 08 19:41:17.593865 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:17.593831 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd7775959-dbrvn_df3f5e05-9c06-4511-90c2-21dd0dd58469/manager/0.log" Jul 08 19:41:17.610448 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:17.610390 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dd7775959-dbrvn_df3f5e05-9c06-4511-90c2-21dd0dd58469/copy-manifests/0.log" Jul 08 19:41:19.059204 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:19.059167 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bfc697457-ckzq4_96c4e6df-17ed-4e81-b6d4-505ad41438ba/manager/0.log" Jul 08 19:41:19.114294 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:19.114255 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-p9664_93c53334-3f45-474f-879e-85f36ebfa324/openshift-lws-operator/0.log" Jul 08 19:41:19.800739 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:19.800698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85bd9454df-q9k5c_f107e6dd-9b2b-42a4-b186-04b4b2f778a5/kube-auth-proxy/0.log" Jul 08 19:41:19.933926 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:19.933898 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cf956c5dc-r496w_125a062b-4f51-4040-bb38-f0a8a610d369/router/0.log" Jul 08 19:41:21.322778 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:21.322746 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85bd9454df-q9k5c_f107e6dd-9b2b-42a4-b186-04b4b2f778a5/kube-auth-proxy/0.log" Jul 08 19:41:21.458161 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:21.458088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cf956c5dc-r496w_125a062b-4f51-4040-bb38-f0a8a610d369/router/0.log" Jul 08 19:41:24.503641 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:24.503603 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-llkzl_a86510c6-35a5-48f4-badb-aea0921624d9/kube-storage-version-migrator-operator/1.log" Jul 08 19:41:24.505210 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:24.505177 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-llkzl_a86510c6-35a5-48f4-badb-aea0921624d9/kube-storage-version-migrator-operator/0.log" Jul 08 19:41:25.655924 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.655897 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/kube-multus-additional-cni-plugins/0.log" Jul 08 19:41:25.677197 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.677171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/egress-router-binary-copy/0.log" Jul 08 19:41:25.698413 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.698387 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/cni-plugins/0.log" Jul 08 19:41:25.720232 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.720212 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/bond-cni-plugin/0.log" Jul 08 19:41:25.740601 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.740572 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/routeoverride-cni/0.log" Jul 08 19:41:25.762690 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.762671 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/whereabouts-cni-bincopy/0.log" Jul 08 19:41:25.784389 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.784369 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hdhlq_75397fac-717f-42a0-a024-70a29cc0357d/whereabouts-cni/0.log" Jul 08 19:41:25.996048 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:25.996016 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qjc6t_ed7fec0f-84e0-419e-8d60-f32fb07cd23c/kube-multus/0.log" Jul 08 19:41:26.079368 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:26.079330 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gmnq_7578c673-39a4-4942-9470-e85a33f20c0e/network-metrics-daemon/0.log" Jul 08 19:41:26.098362 ip-10-0-142-223 kubenswrapper[2580]: I0708 19:41:26.098331 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gmnq_7578c673-39a4-4942-9470-e85a33f20c0e/kube-rbac-proxy/0.log"