Jul 10 14:32:54.698321 ip-10-0-136-92 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 14:32:54.698338 ip-10-0-136-92 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 14:32:54.698347 ip-10-0-136-92 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 14:32:54.698697 ip-10-0-136-92 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 14:33:04.794050 ip-10-0-136-92 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 14:33:04.794065 ip-10-0-136-92 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ad97dab6160c4377aa8983ba3fe46f9e -- Jul 10 14:35:10.288519 ip-10-0-136-92 systemd[1]: Starting Kubernetes Kubelet... Jul 10 14:35:10.709179 ip-10-0-136-92 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:35:10.709179 ip-10-0-136-92 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 14:35:10.709179 ip-10-0-136-92 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:35:10.709179 ip-10-0-136-92 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 14:35:10.709179 ip-10-0-136-92 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 14:35:10.710398 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.710091 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 14:35:10.714452 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714435 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714453 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714460 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714464 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714469 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714475 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714478 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714481 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714484 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714487 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714490 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.714491 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714494 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714549 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714604 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714610 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714615 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714620 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714625 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714629 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714635 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714639 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714644 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714648 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714653 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714658 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714662 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.714856 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.714671 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.715624 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715575 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.715624 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715602 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.715624 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715609 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.715624 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715614 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.715826 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715619 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.715826 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715675 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715716 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715871 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715877 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715881 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715884 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715888 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715891 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715894 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715896 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715899 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715902 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715905 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715907 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715910 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715913 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715916 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715920 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.715909 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715923 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715926 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715929 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715931 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715935 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715940 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715942 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715945 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715947 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715950 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715953 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715956 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715963 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715966 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715969 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715972 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715974 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715977 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715979 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.716429 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715982 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715987 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715990 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715992 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715995 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.715997 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716000 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716003 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716005 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716008 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716011 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716013 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716017 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716019 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716023 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716026 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716028 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716031 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716033 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716036 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.716877 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716038 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716041 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716043 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716046 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716049 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716052 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716055 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716057 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716060 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716062 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716858 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716865 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716868 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716871 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716874 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716876 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716879 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716881 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716885 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716888 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.717388 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716890 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716893 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716896 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716898 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716901 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716903 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716906 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716908 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716911 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716913 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716916 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716919 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716921 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716924 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716928 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716931 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716933 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716936 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716939 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716941 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.717901 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716945 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716948 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716951 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716953 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716956 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716959 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716961 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716964 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716966 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716969 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716972 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716975 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716977 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716979 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716982 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716985 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716987 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716991 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716995 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.718416 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.716998 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717000 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717003 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717005 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717008 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717010 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717015 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717020 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717024 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717027 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717030 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717033 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717036 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717040 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717043 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717045 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717048 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717050 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717053 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.718864 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717056 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717058 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717060 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717063 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717066 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717069 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717071 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717073 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717076 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717078 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717081 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717084 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717086 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717089 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717091 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717094 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717096 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717099 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717101 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717104 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.719338 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717106 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717109 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717111 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717114 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717116 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717119 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717121 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717125 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717127 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717130 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717132 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717542 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717548 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717550 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717553 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717556 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717559 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717562 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717564 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717567 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.719807 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717569 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717572 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717575 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717577 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717580 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717583 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717585 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717588 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717591 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717593 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717596 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717598 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717601 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717603 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717606 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717609 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717611 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717613 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717616 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717619 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.720304 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717621 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717624 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717626 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717629 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717631 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717634 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717636 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717639 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717642 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717645 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717647 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717650 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717655 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717659 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717663 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717666 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717669 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717672 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717674 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.720779 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717680 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717683 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717685 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717688 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717691 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717694 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717696 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717699 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717702 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717705 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717708 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717710 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717713 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717717 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717719 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717722 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717724 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717727 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717730 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717733 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.721268 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717735 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717738 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717741 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717745 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717748 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717750 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717753 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717756 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717758 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717761 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717765 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717768 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717772 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717775 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717778 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717780 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717783 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717786 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717788 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717791 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.721775 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717793 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717796 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717798 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717801 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717803 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717806 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717809 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717811 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717814 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717816 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.717819 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717893 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717902 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717908 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717912 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717916 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717919 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717923 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717927 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717931 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717934 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 14:35:10.722279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717937 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717940 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717943 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717947 2572 flags.go:64] FLAG: --cgroup-root="" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717949 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717952 2572 flags.go:64] FLAG: --client-ca-file="" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717955 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717958 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717963 2572 flags.go:64] FLAG: --cluster-domain="" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717965 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717968 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717971 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717974 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717978 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717981 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717984 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717987 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717990 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717994 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.717996 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718000 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718002 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718011 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718015 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718018 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 14:35:10.722810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718021 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718024 2572 flags.go:64] FLAG: --enable-server="true" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718027 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718031 2572 flags.go:64] FLAG: --event-burst="100" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718034 2572 flags.go:64] FLAG: --event-qps="50" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718037 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718040 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718043 2572 flags.go:64] FLAG: --eviction-hard="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718047 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718049 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718054 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718057 2572 flags.go:64] FLAG: --eviction-soft="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718060 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718062 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718065 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718068 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718071 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718073 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718076 2572 flags.go:64] FLAG: --feature-gates="" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718080 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718082 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718085 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718088 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718091 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718094 2572 flags.go:64] FLAG: --help="false" Jul 10 14:35:10.723439 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718097 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-136-92.ec2.internal" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718100 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718103 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718106 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718109 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718113 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718116 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718119 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718121 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718124 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718127 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718129 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718132 2572 flags.go:64] FLAG: --kube-reserved="" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718135 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718137 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718140 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718143 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718147 2572 flags.go:64] FLAG: --lock-file="" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718150 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718153 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718155 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718160 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718163 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718166 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 14:35:10.724054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718169 2572 flags.go:64] FLAG: --logging-format="text" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718171 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718174 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718177 2572 flags.go:64] FLAG: --manifest-url="" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718180 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718184 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718187 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718204 2572 flags.go:64] FLAG: --max-pods="110" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718207 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718210 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718214 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718217 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718219 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718224 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718227 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718233 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718236 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718239 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718242 2572 flags.go:64] FLAG: --pod-cidr="" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718245 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718250 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718252 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718256 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718258 2572 flags.go:64] FLAG: --port="10250" Jul 10 14:35:10.724666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718261 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718264 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08defd55cbb47b4ea" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718268 2572 flags.go:64] FLAG: --qos-reserved="" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718271 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718274 2572 flags.go:64] FLAG: --register-node="true" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718277 2572 flags.go:64] FLAG: --register-with-taints="" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718280 2572 flags.go:64] FLAG: --registry-burst="10" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718283 2572 flags.go:64] FLAG: --registry-qps="5" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718285 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718288 2572 flags.go:64] FLAG: --reserved-memory="" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718292 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718295 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718297 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718300 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718303 2572 flags.go:64] FLAG: --runonce="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718306 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718309 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718312 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718314 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718318 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718321 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718326 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718328 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718331 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718334 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 14:35:10.725275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718336 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718339 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718342 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718345 2572 flags.go:64] FLAG: --system-cgroups="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718348 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718353 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718356 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718358 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718363 2572 flags.go:64] FLAG: --tls-min-version="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718367 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718370 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718373 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718375 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718378 2572 flags.go:64] FLAG: --v="2" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718382 2572 flags.go:64] FLAG: --version="false" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718386 2572 flags.go:64] FLAG: --vmodule="" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718390 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.718393 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718515 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718518 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718521 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718524 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718526 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.725918 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718531 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718534 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718537 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718540 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718544 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718548 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718551 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718554 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718557 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718559 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718562 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718564 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718567 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718569 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718572 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718574 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718577 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718579 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718583 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718586 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.726522 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718589 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718591 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718594 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718596 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718599 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718603 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718606 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718609 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718611 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718614 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718617 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718619 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718622 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718624 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718627 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718629 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718632 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718636 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718638 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.726998 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718642 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718646 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718649 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718652 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718654 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718657 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718659 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718662 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718664 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718667 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718669 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718673 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718676 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718678 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718680 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718683 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718686 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718688 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718690 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718693 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.727565 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718695 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718698 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718701 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718703 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718706 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718708 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718710 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718713 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718715 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718718 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718722 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718725 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718727 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718730 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718732 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718734 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718737 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718739 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718741 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718744 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.728044 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718746 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718749 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718751 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718756 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718758 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718761 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718763 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718765 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718768 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718770 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718773 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718775 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718777 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718780 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.718782 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.728533 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.719271 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.726435 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.726451 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.726471 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.726477 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726532 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726537 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726541 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726544 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726547 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726550 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726553 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726556 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726558 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726561 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726563 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726566 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726569 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726571 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726574 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.728898 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726577 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726579 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726581 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726584 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726587 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726589 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726592 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726594 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726597 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726599 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726601 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726604 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726606 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726609 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726612 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726615 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726618 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726620 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726623 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726625 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.729382 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726628 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726632 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726637 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726640 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726643 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726646 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726648 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726651 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726653 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726656 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726658 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726661 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726663 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726666 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726669 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726672 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726676 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726679 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726682 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.729891 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726685 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726687 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726690 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726693 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726696 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726698 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726701 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726704 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726707 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726710 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726712 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726715 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726717 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726719 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726722 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726725 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726729 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726732 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726735 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726738 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.730360 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726740 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726743 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726745 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726748 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726751 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726753 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726756 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726758 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726761 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726763 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726766 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726768 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726771 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726773 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726776 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726778 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726781 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726783 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726785 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726789 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.730840 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726792 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726794 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726797 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726799 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726802 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.726807 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726930 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726934 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726937 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726940 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726943 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726945 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726948 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726951 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726953 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 14:35:10.731387 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726956 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726958 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726961 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726963 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726966 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726968 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726971 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726974 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726976 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726979 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726981 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726984 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726986 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726989 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726993 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726996 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.726999 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727002 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727005 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727008 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 14:35:10.731761 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727011 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727013 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727016 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727018 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727021 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727023 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727026 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727029 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727031 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727034 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727036 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727039 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727041 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727044 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727046 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727049 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727051 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727054 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727056 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 14:35:10.732256 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727058 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727061 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727063 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727066 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727069 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727072 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727074 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727076 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727079 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727081 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727084 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727086 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727089 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727092 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727096 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727098 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727101 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727103 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727106 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727109 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 14:35:10.732708 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727112 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727115 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727117 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727120 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727122 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727125 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727127 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727130 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727132 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727135 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727138 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727141 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727144 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727147 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727150 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727153 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727156 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727158 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727161 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727163 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 14:35:10.733184 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727166 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727168 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727171 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727173 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727175 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727178 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727180 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727183 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727186 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727203 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:10.727208 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.727215 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.727327 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.729657 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 14:35:10.733686 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.732271 2572 server.go:1019] "Starting client certificate rotation" Jul 10 14:35:10.734022 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.732366 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 14:35:10.734022 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.733071 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 14:35:10.735251 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.735203 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 14:35:10.736460 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.736439 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 14:35:10.748710 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.748688 2572 log.go:25] "Validated CRI v1 runtime API" Jul 10 14:35:10.750923 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.750909 2572 log.go:25] "Validated CRI v1 image API" Jul 10 14:35:10.751692 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.751670 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 14:35:10.756703 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.756683 2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a84940d5-1eb9-462c-9261-524ea8a045bf:/dev/nvme0n1p3 fd52194f-7b28-491e-896f-771ca4da613e:/dev/nvme0n1p4] Jul 10 14:35:10.756777 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.756702 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 14:35:10.762554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.762435 2572 manager.go:216] Machine: {Timestamp:2026-07-10 14:35:10.760367255 +0000 UTC m=+0.354722765 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102529 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ce3eec557e794159c73975a56ad2a SystemUUID:ec2ce3ee-c557-e794-159c-73975a56ad2a BootID:ad97dab6-160c-4377-aa89-83ba3fe46f9e Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:a9:dd:51:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:a9:dd:51:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:61:8a:17:0a:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 14:35:10.762554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.762547 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 14:35:10.762671 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.762639 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 14:35:10.765300 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.765272 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 14:35:10.765434 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.765304 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 14:35:10.766110 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.766100 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 14:35:10.766147 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.766114 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 14:35:10.766147 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.766126 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 14:35:10.766777 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.766766 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 14:35:10.767593 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.767583 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 14:35:10.770478 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770464 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 10 14:35:10.770798 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770787 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 14:35:10.770838 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770824 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 10 14:35:10.770872 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770840 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 14:35:10.770872 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770865 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 14:35:10.770922 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770878 2572 kubelet.go:398] "Adding apiserver pod source" Jul 10 14:35:10.770922 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.770891 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 14:35:10.771349 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.771336 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 14:35:10.771403 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.771355 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 14:35:10.772280 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.772261 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 14:35:10.773695 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.773678 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 14:35:10.775068 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775054 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 14:35:10.775115 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775085 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 14:35:10.775302 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775292 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 14:35:10.775333 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775307 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 14:35:10.775333 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775313 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 14:35:10.775333 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775319 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 14:35:10.775333 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775327 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 14:35:10.775333 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775333 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 14:35:10.775820 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775812 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 14:35:10.775850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775820 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 14:35:10.775850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775828 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 14:35:10.775850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775833 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 14:35:10.775850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775847 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 14:35:10.775951 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.775855 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 14:35:10.777228 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.777218 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 14:35:10.777228 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.777228 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 14:35:10.780431 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.780407 2572 server.go:1268] "Started kubelet" Jul 10 14:35:10.781297 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.780510 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 14:35:10.781297 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.780578 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 14:35:10.781297 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.780623 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 14:35:10.781551 ip-10-0-136-92 systemd[1]: Started Kubernetes Kubelet. Jul 10 14:35:10.783997 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.783967 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 14:35:10.785353 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.785332 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 10 14:35:10.788206 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.788167 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 14:35:10.788699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.788345 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 14:35:10.788699 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.788411 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 14:35:10.788699 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.788439 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 14:35:10.790059 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790042 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 14:35:10.790137 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790043 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 14:35:10.790219 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.790136 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:10.790792 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790772 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 14:35:10.790868 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790773 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 14:35:10.790868 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790806 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 14:35:10.791137 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790873 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 10 14:35:10.791137 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790885 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 10 14:35:10.791137 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790938 2572 factory.go:55] Registering systemd factory Jul 10 14:35:10.791137 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.790951 2572 factory.go:223] Registration of the systemd container factory successfully Jul 10 14:35:10.791310 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791150 2572 factory.go:153] Registering CRI-O factory Jul 10 14:35:10.791310 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791165 2572 factory.go:223] Registration of the crio container factory successfully Jul 10 14:35:10.791310 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791224 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 14:35:10.791310 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791243 2572 factory.go:103] Registering Raw factory Jul 10 14:35:10.791310 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791253 2572 manager.go:1196] Started watching for new ooms in manager Jul 10 14:35:10.791753 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.791739 2572 manager.go:349] Starting recovery of all containers Jul 10 14:35:10.792916 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.792889 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 14:35:10.792999 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.792958 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 14:35:10.793964 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.793067 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-92.ec2.internal.18c0f457e50554b7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-92.ec2.internal,UID:ip-10-0-136-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-92.ec2.internal,},FirstTimestamp:2026-07-10 14:35:10.780380343 +0000 UTC m=+0.374735853,LastTimestamp:2026-07-10 14:35:10.780380343 +0000 UTC m=+0.374735853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-92.ec2.internal,}" Jul 10 14:35:10.802372 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.802172 2572 manager.go:354] Recovery completed Jul 10 14:35:10.804861 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.804822 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8bqqh" Jul 10 14:35:10.807471 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.807452 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:10.811837 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.811821 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:10.811909 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.811849 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:10.811909 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.811867 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:10.812750 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.812736 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 14:35:10.812750 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.812747 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 14:35:10.812891 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.812770 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 10 14:35:10.812891 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.812772 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:10.813819 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.813804 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:10.813888 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.813835 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:10.813888 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.813850 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:10.814059 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.814045 2572 policy_none.go:49] "None policy: Start" Jul 10 14:35:10.814059 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.814062 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 14:35:10.814169 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.814071 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 14:35:10.814396 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.814319 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-92.ec2.internal.18c0f457e6e54d41 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-92.ec2.internal,UID:ip-10-0-136-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-92.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-92.ec2.internal,},FirstTimestamp:2026-07-10 14:35:10.811835713 +0000 UTC m=+0.406191223,LastTimestamp:2026-07-10 14:35:10.811835713 +0000 UTC m=+0.406191223,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-92.ec2.internal,}" Jul 10 14:35:10.815112 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.815102 2572 policy_none.go:47] "Start" Jul 10 14:35:10.816574 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.816559 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8bqqh" Jul 10 14:35:10.857838 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.857817 2572 manager.go:337] "Starting Device Plugin manager" Jul 10 14:35:10.857965 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.857855 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 14:35:10.857965 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.857865 2572 server.go:87] "Starting device plugin registration server" Jul 10 14:35:10.858143 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.858132 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 14:35:10.858212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.858145 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 14:35:10.858277 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.858259 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 14:35:10.858355 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.858340 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 14:35:10.858355 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.858354 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 14:35:10.858962 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.858935 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 14:35:10.859062 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.859003 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:10.958480 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.958442 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:10.959585 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.959549 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:10.959585 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.959584 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:10.959681 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.959594 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:10.959681 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.959619 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-92.ec2.internal" Jul 10 14:35:10.970428 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:10.970408 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-92.ec2.internal" Jul 10 14:35:10.970493 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.970433 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-92.ec2.internal\": node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:10.994794 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:10.994766 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.068655 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.068623 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 14:35:11.070056 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.070038 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 14:35:11.070154 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.070082 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 14:35:11.070154 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.070111 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 14:35:11.070274 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.070152 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 14:35:11.072584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.072560 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:35:11.095018 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.094988 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.171247 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.171185 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal"] Jul 10 14:35:11.171355 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.171294 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:11.172347 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.172332 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:11.172401 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.172373 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:11.172401 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.172387 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:11.173933 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.173921 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:11.174112 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174100 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.174145 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174128 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:11.174635 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174619 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:11.174706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174651 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:11.174706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174658 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:11.174706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174666 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:11.174706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174691 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:11.174706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.174704 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:11.175995 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.175982 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.176047 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.176006 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 14:35:11.176707 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.176691 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientMemory" Jul 10 14:35:11.176801 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.176723 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 14:35:11.176801 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.176737 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeHasSufficientPID" Jul 10 14:35:11.192172 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.192144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.192172 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.192175 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.192394 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.192213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fef107a90282d0eb2837214ea08a54d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-92.ec2.internal\" (UID: \"9fef107a90282d0eb2837214ea08a54d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.192905 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.192888 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-92.ec2.internal\" not found" node="ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.195079 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.195058 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.197001 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.196984 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-92.ec2.internal\" not found" node="ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293060 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293029 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293152 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293035 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293152 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293099 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293152 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fef107a90282d0eb2837214ea08a54d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-92.ec2.internal\" (UID: \"9fef107a90282d0eb2837214ea08a54d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293318 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293182 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fef107a90282d0eb2837214ea08a54d-config\") pod \"kube-apiserver-proxy-ip-10-0-136-92.ec2.internal\" (UID: \"9fef107a90282d0eb2837214ea08a54d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.293318 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.293225 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62f94fe7fee4115957dc1c2ab5f235ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal\" (UID: \"62f94fe7fee4115957dc1c2ab5f235ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.295332 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.295314 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.396343 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.396308 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.495067 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.495023 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.497363 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.497337 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.500668 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.500650 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.598288 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.598180 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.699133 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.699099 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.731513 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.731474 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 14:35:11.732115 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.731664 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 14:35:11.799871 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:11.799841 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-92.ec2.internal\" not found" Jul 10 14:35:11.820609 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.820566 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 14:30:11 +0000 UTC" deadline="2028-04-26 21:08:48.737036408 +0000 UTC" Jul 10 14:35:11.820609 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.820606 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15750h33m36.916434329s" Jul 10 14:35:11.848418 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.848356 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:35:11.891296 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.891260 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.912958 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.912929 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 14:35:11.914707 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.914688 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" Jul 10 14:35:11.931528 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:11.931505 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 14:35:12.066311 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:12.066275 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f94fe7fee4115957dc1c2ab5f235ab.slice/crio-12b1c75f18b8d3665c170483bc0e5705725c85dd69e9815ada8d184bb9036138 WatchSource:0}: Error finding container 12b1c75f18b8d3665c170483bc0e5705725c85dd69e9815ada8d184bb9036138: Status 404 returned error can't find the container with id 12b1c75f18b8d3665c170483bc0e5705725c85dd69e9815ada8d184bb9036138 Jul 10 14:35:12.067007 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:12.066988 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fef107a90282d0eb2837214ea08a54d.slice/crio-dd59f03a5e783a7f31c03856e5bce19112acbe4f769355db396ed3c0de31226b WatchSource:0}: Error finding container dd59f03a5e783a7f31c03856e5bce19112acbe4f769355db396ed3c0de31226b: Status 404 returned error can't find the container with id dd59f03a5e783a7f31c03856e5bce19112acbe4f769355db396ed3c0de31226b Jul 10 14:35:12.072600 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.072580 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:35:12.073615 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.073498 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" event={"ID":"9fef107a90282d0eb2837214ea08a54d","Type":"ContainerStarted","Data":"dd59f03a5e783a7f31c03856e5bce19112acbe4f769355db396ed3c0de31226b"} Jul 10 14:35:12.074777 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.074748 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" event={"ID":"62f94fe7fee4115957dc1c2ab5f235ab","Type":"ContainerStarted","Data":"12b1c75f18b8d3665c170483bc0e5705725c85dd69e9815ada8d184bb9036138"} Jul 10 14:35:12.188711 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.188627 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:35:12.358721 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.358685 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:35:12.580964 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.580881 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 14:35:12.774155 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.774116 2572 apiserver.go:52] "Watching apiserver" Jul 10 14:35:12.783688 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.783657 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 14:35:12.784604 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.784178 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-m4kfh","openshift-network-operator/iptables-alerter-7rqhp","openshift-ovn-kubernetes/ovnkube-node-l6ccb","kube-system/global-pull-secret-syncer-lsnd7","kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9pz7s","openshift-dns/node-resolver-txdwq","openshift-image-registry/node-ca-r6trl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal","openshift-multus/multus-additional-cni-plugins-n6hkm","openshift-network-diagnostics/network-check-target-hftj9","kube-system/konnectivity-agent-x55kl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7","openshift-multus/multus-d28x6"] Jul 10 14:35:12.786253 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.786221 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.787929 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.787674 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:12.787929 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.787743 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:12.789413 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.789388 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.790581 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.790556 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.790690 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.790668 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qj582\"" Jul 10 14:35:12.790772 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.790566 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.790875 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.790823 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:12.790949 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.790929 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.791109 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.791090 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 14:35:12.791784 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.791761 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.792532 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.792369 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 14:35:12.792977 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.792960 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 14:35:12.794125 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.793726 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.794444 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.794412 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fftxb\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795107 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795340 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795348 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795403 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795565 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795580 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795626 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795699 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795750 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mqn5s\"" Jul 10 14:35:12.795928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.795806 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 14:35:12.796623 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.796604 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.796894 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.796858 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.797255 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.797215 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.797938 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.797917 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4rzv\"" Jul 10 14:35:12.798322 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.798302 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 14:35:12.798966 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.798812 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.799067 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.799001 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.799265 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.799234 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vz2hb\"" Jul 10 14:35:12.799750 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.799242 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:12.800658 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.800637 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.800801 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.800763 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.801163 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.801144 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fkxmq\"" Jul 10 14:35:12.801363 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.801230 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 14:35:12.801624 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.801612 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.802282 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.802263 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:12.802380 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.802362 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:12.802745 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.802723 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 14:35:12.802836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.802817 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:35:12.802898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.802878 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wbbdw\"" Jul 10 14:35:12.802898 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.802816 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 14:35:12.803764 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.803701 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:12.803879 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.803831 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-ovn\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.803862 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-env-overrides\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804455 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-os-release\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804484 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-daemon-config\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-multus-certs\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804536 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-smk2c\"" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flpsc\" (UniqueName: \"kubernetes.io/projected/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-kube-api-access-flpsc\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-slash\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-kubelet-config\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804676 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cni-binary-copy\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-bin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804737 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8596l\" (UniqueName: \"kubernetes.io/projected/1ba93023-66ea-493f-80b8-ec6119aa9c36-kube-api-access-8596l\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804775 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-var-lib-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804806 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804839 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovn-node-metrics-cert\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804876 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b224fd08-5b50-4b09-8c9c-30e22d27b052-tmp-dir\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804899 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwxc4\" (UniqueName: \"kubernetes.io/projected/b224fd08-5b50-4b09-8c9c-30e22d27b052-kube-api-access-jwxc4\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.807294 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804929 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-kubelet\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804965 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-systemd-units\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.804998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-bin\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.805030 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-netd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.805488 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806203 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806398 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-k8s-cni-cncf-io\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806461 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-netns\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806502 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-systemd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806536 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-etc-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806571 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806604 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-multus\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806631 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-conf-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-etc-kubernetes\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806707 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-netns\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806756 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-log-socket\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806800 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806860 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-system-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808160 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806923 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-socket-dir-parent\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806961 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-kubelet\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.806992 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-hostroot\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807026 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b224fd08-5b50-4b09-8c9c-30e22d27b052-hosts-file\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807056 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-config\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807118 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-script-lib\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807140 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807161 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-node-log\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807199 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-dbus\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807231 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.807283 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cnibin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.808896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.808086 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8vntx\"" Jul 10 14:35:12.811890 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.811871 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 14:35:12.812086 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.812070 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 14:35:12.812531 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.812274 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 14:35:12.876526 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.876499 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ncz62" Jul 10 14:35:12.902686 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.902644 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ncz62" Jul 10 14:35:12.903122 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.903100 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 14:35:12.907621 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907593 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-netns\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.907723 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.907723 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907698 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-netns\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.907848 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907737 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-system-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.907848 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907758 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-hostroot\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.907848 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.907762 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:12.907848 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907789 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-hostroot\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.907848 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907820 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-system-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.907859 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:13.407814764 +0000 UTC m=+3.002170265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907875 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-sys-fs\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907900 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907934 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907952 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-config\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brzwd\" (UniqueName: \"kubernetes.io/projected/0dbbca7d-f975-49ff-8726-92bcd15c02bd-kube-api-access-brzwd\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907985 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-kubernetes\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.907999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-tuned\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908014 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8sk6\" (UniqueName: \"kubernetes.io/projected/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-kube-api-access-b8sk6\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:12.908037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908047 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-node-log\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908050 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908082 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-node-log\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908143 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cnibin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908175 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-tmp\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908193 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908232 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-os-release\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908236 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cnibin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-daemon-config\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908265 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flpsc\" (UniqueName: \"kubernetes.io/projected/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-kube-api-access-flpsc\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908376 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dbbca7d-f975-49ff-8726-92bcd15c02bd-iptables-alerter-script\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908409 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-systemd\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908440 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-slash\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908469 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-kubelet-config\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908505 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-os-release\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.908523 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908531 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cni-binary-copy\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908549 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk9jn\" (UniqueName: \"kubernetes.io/projected/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-kube-api-access-wk9jn\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908567 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-os-release\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908610 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-kubelet-config\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908650 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-daemon-config\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908677 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-config\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908690 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cnibin\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-kubelet\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908726 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-slash\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908752 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908807 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-bin\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908822 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-kubelet\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908860 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-conf-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-etc-kubernetes\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908912 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-conf-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908926 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-bin\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909101 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908933 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58wzc\" (UniqueName: \"kubernetes.io/projected/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kube-api-access-58wzc\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908957 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-cni-binary-copy\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908963 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysconfig\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.908969 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-etc-kubernetes\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909001 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909019 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-konnectivity-ca\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909036 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-etc-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909072 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-multus\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909075 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-etc-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909087 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-log-socket\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909117 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-log-socket\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909117 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909122 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-multus\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909164 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-socket-dir-parent\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-kubelet\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909234 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-socket-dir-parent\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909240 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57c89553-a981-4c32-8289-e82919822315-host\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:12.909706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909255 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57c89553-a981-4c32-8289-e82919822315-serviceca\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909274 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-socket-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909296 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-kubelet\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909298 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-modprobe-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909324 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-run\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909341 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b224fd08-5b50-4b09-8c9c-30e22d27b052-hosts-file\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909357 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-script-lib\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909378 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-var-lib-kubelet\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909396 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909412 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5gtr\" (UniqueName: \"kubernetes.io/projected/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-kube-api-access-d5gtr\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909424 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b224fd08-5b50-4b09-8c9c-30e22d27b052-hosts-file\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-dbus\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909481 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909487 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-multus-cni-dir\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909497 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-ovn\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909517 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-ovn\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-env-overrides\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909553 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-multus-certs\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-device-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909578 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0556a41c-eefb-467e-91bb-209b5bf6d33d-dbus\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909607 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-multus-certs\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909618 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-sys\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909678 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-bin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909705 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-conf\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909732 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-agent-certs\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909760 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8596l\" (UniqueName: \"kubernetes.io/projected/1ba93023-66ea-493f-80b8-ec6119aa9c36-kube-api-access-8596l\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909783 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-var-lib-cni-bin\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909788 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-var-lib-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909817 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovn-node-metrics-cert\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909826 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-var-lib-openvswitch\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909871 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-env-overrides\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.910984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909924 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovnkube-script-lib\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.909952 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-lib-modules\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910041 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-host\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910075 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-system-cni-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910093 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b224fd08-5b50-4b09-8c9c-30e22d27b052-tmp-dir\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910108 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwxc4\" (UniqueName: \"kubernetes.io/projected/b224fd08-5b50-4b09-8c9c-30e22d27b052-kube-api-access-jwxc4\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910123 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-systemd-units\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910138 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-netd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910157 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-k8s-cni-cncf-io\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910176 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-netns\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910219 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-systemd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910237 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59j6b\" (UniqueName: \"kubernetes.io/projected/57c89553-a981-4c32-8289-e82919822315-kube-api-access-59j6b\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910252 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-registration-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910255 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-host-cni-netd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dbbca7d-f975-49ff-8726-92bcd15c02bd-host-slash\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910274 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910338 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-k8s-cni-cncf-io\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910367 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-host-run-netns\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.911751 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910395 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-run-systemd\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.912433 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910407 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b224fd08-5b50-4b09-8c9c-30e22d27b052-tmp-dir\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.912433 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.910416 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ba93023-66ea-493f-80b8-ec6119aa9c36-systemd-units\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.913840 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.913821 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ba93023-66ea-493f-80b8-ec6119aa9c36-ovn-node-metrics-cert\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:12.918811 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.918792 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:12.918909 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.918816 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:12.918909 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.918830 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:12.918909 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:12.918891 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:13.41887222 +0000 UTC m=+3.013227738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:12.922349 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.922325 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwxc4\" (UniqueName: \"kubernetes.io/projected/b224fd08-5b50-4b09-8c9c-30e22d27b052-kube-api-access-jwxc4\") pod \"node-resolver-txdwq\" (UID: \"b224fd08-5b50-4b09-8c9c-30e22d27b052\") " pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:12.922437 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.922385 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flpsc\" (UniqueName: \"kubernetes.io/projected/dc7ceef6-890f-4ff7-93ec-f2568a64d3cb-kube-api-access-flpsc\") pod \"multus-d28x6\" (UID: \"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb\") " pod="openshift-multus/multus-d28x6" Jul 10 14:35:12.922501 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:12.922465 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8596l\" (UniqueName: \"kubernetes.io/projected/1ba93023-66ea-493f-80b8-ec6119aa9c36-kube-api-access-8596l\") pod \"ovnkube-node-l6ccb\" (UID: \"1ba93023-66ea-493f-80b8-ec6119aa9c36\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:13.010669 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010614 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk9jn\" (UniqueName: \"kubernetes.io/projected/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-kube-api-access-wk9jn\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.010669 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010666 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-os-release\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010696 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cnibin\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58wzc\" (UniqueName: \"kubernetes.io/projected/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kube-api-access-58wzc\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysconfig\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010798 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-konnectivity-ca\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57c89553-a981-4c32-8289-e82919822315-host\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57c89553-a981-4c32-8289-e82919822315-serviceca\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-socket-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010874 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-modprobe-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010890 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-run\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010911 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-var-lib-kubelet\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.010928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010927 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010946 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5gtr\" (UniqueName: \"kubernetes.io/projected/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-kube-api-access-d5gtr\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010968 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.010995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-device-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011030 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-sys\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011054 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-conf\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-agent-certs\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011130 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-lib-modules\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011153 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-host\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011177 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-system-cni-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011210 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59j6b\" (UniqueName: \"kubernetes.io/projected/57c89553-a981-4c32-8289-e82919822315-kube-api-access-59j6b\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011235 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-registration-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011278 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dbbca7d-f975-49ff-8726-92bcd15c02bd-host-slash\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011324 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-sys-fs\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011351 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.011584 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brzwd\" (UniqueName: \"kubernetes.io/projected/0dbbca7d-f975-49ff-8726-92bcd15c02bd-kube-api-access-brzwd\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011404 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-kubernetes\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011430 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-tuned\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011454 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8sk6\" (UniqueName: \"kubernetes.io/projected/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-kube-api-access-b8sk6\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-tmp\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011524 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011579 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dbbca7d-f975-49ff-8726-92bcd15c02bd-iptables-alerter-script\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011605 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-systemd\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011711 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-systemd\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011777 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011831 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-device-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011882 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-os-release\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011925 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cnibin\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.011970 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.012022 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-sys\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.012139 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysconfig\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.012514 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.012220 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.012320 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:13.512298989 +0000 UTC m=+3.106654504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.012633 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.012796 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-sysctl-conf\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.012972 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-konnectivity-ca\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013028 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57c89553-a981-4c32-8289-e82919822315-host\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.013366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013306 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-var-lib-kubelet\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.013666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013378 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-socket-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.013666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013435 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57c89553-a981-4c32-8289-e82919822315-serviceca\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.013666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013545 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.013666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013563 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-modprobe-d\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.013831 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013747 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-kubernetes\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.013944 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013916 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.013944 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.013938 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.014105 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014083 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-lib-modules\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.014212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014102 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.014212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014155 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-host\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.014212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014159 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-registration-dir\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.014212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014189 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dbbca7d-f975-49ff-8726-92bcd15c02bd-host-slash\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.014212 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014208 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-system-cni-dir\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.014627 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014233 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-run\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.014627 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014237 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-sys-fs\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.014734 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.014662 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dbbca7d-f975-49ff-8726-92bcd15c02bd-iptables-alerter-script\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.016096 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.016048 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cb7717f-6743-4d0d-b398-1a32ab7bd6e6-agent-certs\") pod \"konnectivity-agent-x55kl\" (UID: \"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6\") " pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:13.016332 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.016313 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-tmp\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.017134 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.017100 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-etc-tuned\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.022830 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.022791 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk9jn\" (UniqueName: \"kubernetes.io/projected/ba4b1c2f-f22a-4ccb-8b8c-5dae48199094-kube-api-access-wk9jn\") pod \"tuned-9pz7s\" (UID: \"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094\") " pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.023188 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.023166 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58wzc\" (UniqueName: \"kubernetes.io/projected/7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1-kube-api-access-58wzc\") pod \"aws-ebs-csi-driver-node-zbfm7\" (UID: \"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.023603 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.023582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5gtr\" (UniqueName: \"kubernetes.io/projected/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-kube-api-access-d5gtr\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:13.028112 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.028090 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brzwd\" (UniqueName: \"kubernetes.io/projected/0dbbca7d-f975-49ff-8726-92bcd15c02bd-kube-api-access-brzwd\") pod \"iptables-alerter-7rqhp\" (UID: \"0dbbca7d-f975-49ff-8726-92bcd15c02bd\") " pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.032263 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.032142 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59j6b\" (UniqueName: \"kubernetes.io/projected/57c89553-a981-4c32-8289-e82919822315-kube-api-access-59j6b\") pod \"node-ca-r6trl\" (UID: \"57c89553-a981-4c32-8289-e82919822315\") " pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.032911 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.032887 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8sk6\" (UniqueName: \"kubernetes.io/projected/97c0fe49-1932-48b3-9f6a-0b4b0d0844fd-kube-api-access-b8sk6\") pod \"multus-additional-cni-plugins-n6hkm\" (UID: \"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd\") " pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.103588 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.103507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-txdwq" Jul 10 14:35:13.117278 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.117014 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:13.126302 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.126264 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:13.131449 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.131423 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d28x6" Jul 10 14:35:13.140865 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.140839 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" Jul 10 14:35:13.147834 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.147807 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r6trl" Jul 10 14:35:13.155927 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.155903 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" Jul 10 14:35:13.168932 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.168904 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7rqhp" Jul 10 14:35:13.173971 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.173943 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" Jul 10 14:35:13.415307 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.415221 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:13.415443 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.415371 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:13.415511 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.415444 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:14.415425324 +0000 UTC m=+4.009780837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:13.515893 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.515856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:13.516057 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.515905 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:13.516057 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516019 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:13.516057 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516035 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:13.516057 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516056 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:13.516313 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516070 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:13.516313 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516084 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:14.516065267 +0000 UTC m=+4.110420768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:13.516313 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:13.516112 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:14.516100222 +0000 UTC m=+4.110455720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:13.905489 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.905384 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:30:12 +0000 UTC" deadline="2028-03-02 21:33:08.076627813 +0000 UTC" Jul 10 14:35:13.905489 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:13.905423 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14430h57m54.171209081s" Jul 10 14:35:14.071136 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.071099 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:14.071342 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.071151 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:14.071342 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.071267 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:14.071470 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.071391 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:14.194477 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.194441 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa67dcc_e1d7_40e2_95a2_ab8c2c8b10e1.slice/crio-23ebc04a6b454be7d9c82bc8f164fb90d70029c467a178682620d77c9295695f WatchSource:0}: Error finding container 23ebc04a6b454be7d9c82bc8f164fb90d70029c467a178682620d77c9295695f: Status 404 returned error can't find the container with id 23ebc04a6b454be7d9c82bc8f164fb90d70029c467a178682620d77c9295695f Jul 10 14:35:14.196171 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.196130 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbbca7d_f975_49ff_8726_92bcd15c02bd.slice/crio-425b4c656e57e6b28ffa7b0d63047d630a51a094e05e0ba369221e54933c49d5 WatchSource:0}: Error finding container 425b4c656e57e6b28ffa7b0d63047d630a51a094e05e0ba369221e54933c49d5: Status 404 returned error can't find the container with id 425b4c656e57e6b28ffa7b0d63047d630a51a094e05e0ba369221e54933c49d5 Jul 10 14:35:14.199749 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.199683 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4b1c2f_f22a_4ccb_8b8c_5dae48199094.slice/crio-8b6e429bd5cb36d9f9fa3e4eb16c7f3649a63c819190586389886bcec4683346 WatchSource:0}: Error finding container 8b6e429bd5cb36d9f9fa3e4eb16c7f3649a63c819190586389886bcec4683346: Status 404 returned error can't find the container with id 8b6e429bd5cb36d9f9fa3e4eb16c7f3649a63c819190586389886bcec4683346 Jul 10 14:35:14.205839 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.205818 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c89553_a981_4c32_8289_e82919822315.slice/crio-32d7d1e0c407f92c933b844f96a229076573933c325bab1a16e9436b18c9219c WatchSource:0}: Error finding container 32d7d1e0c407f92c933b844f96a229076573933c325bab1a16e9436b18c9219c: Status 404 returned error can't find the container with id 32d7d1e0c407f92c933b844f96a229076573933c325bab1a16e9436b18c9219c Jul 10 14:35:14.206190 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.206167 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7ceef6_890f_4ff7_93ec_f2568a64d3cb.slice/crio-1d12ca3015d7b31b66674871aaf4c91557e1d4f5cc963369ac40a6bca757a746 WatchSource:0}: Error finding container 1d12ca3015d7b31b66674871aaf4c91557e1d4f5cc963369ac40a6bca757a746: Status 404 returned error can't find the container with id 1d12ca3015d7b31b66674871aaf4c91557e1d4f5cc963369ac40a6bca757a746 Jul 10 14:35:14.210948 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:14.209369 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb7717f_6743_4d0d_b398_1a32ab7bd6e6.slice/crio-0356f2932b6b39c7e14d714816037abd8811b03ff5f2c617ec29254081f41301 WatchSource:0}: Error finding container 0356f2932b6b39c7e14d714816037abd8811b03ff5f2c617ec29254081f41301: Status 404 returned error can't find the container with id 0356f2932b6b39c7e14d714816037abd8811b03ff5f2c617ec29254081f41301 Jul 10 14:35:14.424062 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.423869 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:14.424179 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.424008 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:14.424255 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.424231 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:16.424182627 +0000 UTC m=+6.018538123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:14.524878 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.524835 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:14.525073 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.524899 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:14.525073 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.524980 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:14.525073 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.525005 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:14.525073 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.525012 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:14.525073 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.525020 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:14.525278 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.525087 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:16.525067494 +0000 UTC m=+6.119423007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:14.525278 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:14.525109 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:16.525099571 +0000 UTC m=+6.119455068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:14.906553 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.906434 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 14:30:12 +0000 UTC" deadline="2028-03-27 19:37:19.361079376 +0000 UTC" Jul 10 14:35:14.906553 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:14.906500 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15029h2m4.454583722s" Jul 10 14:35:15.073607 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.073114 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:15.073607 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:15.073258 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:15.087529 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.087487 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"65ac2dfce7b46396c12c0e421551555304f691e2a148741fe6c568fb5c455745"} Jul 10 14:35:15.090154 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.090100 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" event={"ID":"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094","Type":"ContainerStarted","Data":"8b6e429bd5cb36d9f9fa3e4eb16c7f3649a63c819190586389886bcec4683346"} Jul 10 14:35:15.097210 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.097158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7rqhp" event={"ID":"0dbbca7d-f975-49ff-8726-92bcd15c02bd","Type":"ContainerStarted","Data":"425b4c656e57e6b28ffa7b0d63047d630a51a094e05e0ba369221e54933c49d5"} Jul 10 14:35:15.103169 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.103138 2572 generic.go:356] "Generic (PLEG): container finished" podID="62f94fe7fee4115957dc1c2ab5f235ab" containerID="14fa5b8f2d35ee3839bf8de1a89c0b57b8a6b01d13e5d4d2d79ebd6e9ae32a6a" exitCode=0 Jul 10 14:35:15.103343 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.103239 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" event={"ID":"62f94fe7fee4115957dc1c2ab5f235ab","Type":"ContainerDied","Data":"14fa5b8f2d35ee3839bf8de1a89c0b57b8a6b01d13e5d4d2d79ebd6e9ae32a6a"} Jul 10 14:35:15.124416 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.124377 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-txdwq" event={"ID":"b224fd08-5b50-4b09-8c9c-30e22d27b052","Type":"ContainerStarted","Data":"300437da60bc016f9c35b970b45144409577575c59a28b12fd9885ddde364dc3"} Jul 10 14:35:15.133049 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.132948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x55kl" event={"ID":"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6","Type":"ContainerStarted","Data":"0356f2932b6b39c7e14d714816037abd8811b03ff5f2c617ec29254081f41301"} Jul 10 14:35:15.136769 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.136675 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r6trl" event={"ID":"57c89553-a981-4c32-8289-e82919822315","Type":"ContainerStarted","Data":"32d7d1e0c407f92c933b844f96a229076573933c325bab1a16e9436b18c9219c"} Jul 10 14:35:15.146302 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.146269 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerStarted","Data":"4ee7af32099e51621b0b4ac4687b79f25e58258c31781a0a052caba18214351e"} Jul 10 14:35:15.157386 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.157316 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" event={"ID":"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1","Type":"ContainerStarted","Data":"23ebc04a6b454be7d9c82bc8f164fb90d70029c467a178682620d77c9295695f"} Jul 10 14:35:15.169337 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.169302 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" event={"ID":"9fef107a90282d0eb2837214ea08a54d","Type":"ContainerStarted","Data":"7e777abd6c6653141b6e5deaada1a5adcdecab738afa06036b76cc174985f5df"} Jul 10 14:35:15.192864 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:15.192401 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d28x6" event={"ID":"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb","Type":"ContainerStarted","Data":"1d12ca3015d7b31b66674871aaf4c91557e1d4f5cc963369ac40a6bca757a746"} Jul 10 14:35:16.071130 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.071063 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:16.071612 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.071207 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:16.071612 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.071606 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:16.071752 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.071715 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:16.219383 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.219347 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" event={"ID":"62f94fe7fee4115957dc1c2ab5f235ab","Type":"ContainerStarted","Data":"c607ddc668180d7678b5cb965e027751030c192b5766345cedddcacc4a976963"} Jul 10 14:35:16.237319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.236338 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-92.ec2.internal" podStartSLOduration=4.2363149 podStartE2EDuration="4.2363149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:15.188705178 +0000 UTC m=+4.783060696" watchObservedRunningTime="2026-07-10 14:35:16.2363149 +0000 UTC m=+5.830670430" Jul 10 14:35:16.441107 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.440446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:16.441107 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.440633 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:16.441107 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.440694 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:20.440676459 +0000 UTC m=+10.035031985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.541378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:16.541476 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.541622 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.541643 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.541663 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.541724 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:20.541703607 +0000 UTC m=+10.136059106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.542114 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:16.542213 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:16.542163 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:20.542147784 +0000 UTC m=+10.136503286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:17.073284 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:17.073250 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:17.073730 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:17.073390 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:18.071021 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:18.070985 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:18.071250 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:18.071119 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:18.071339 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:18.071313 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:18.071487 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:18.071452 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:19.071923 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:19.071382 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:19.071923 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:19.071516 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:20.070534 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:20.070501 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:20.070713 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.070660 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:20.071002 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:20.070510 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:20.071138 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.071089 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:20.478366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:20.478272 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:20.478801 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.478479 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:20.478801 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.478547 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:28.478533161 +0000 UTC m=+18.072888657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:20.578823 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:20.578781 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:20.578981 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:20.578854 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:20.578981 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.578969 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:20.579097 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.578979 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:20.579097 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.579004 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:20.579097 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.579018 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:20.579097 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.579034 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:28.579019235 +0000 UTC m=+18.173374735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:20.579097 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:20.579064 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:28.579046086 +0000 UTC m=+18.173401583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:21.071894 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:21.071851 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:21.072083 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:21.071952 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:22.071708 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:22.071120 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:22.071708 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:22.071153 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:22.071708 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:22.071285 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:22.071708 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:22.071671 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:23.070977 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:23.070471 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:23.070977 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:23.070606 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:24.070999 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:24.070381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:24.070999 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:24.070424 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:24.070999 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:24.070531 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:24.070999 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:24.070956 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:25.070745 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:25.070712 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:25.070920 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:25.070841 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:26.070345 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:26.070311 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:26.070681 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:26.070310 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:26.070681 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:26.070430 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:26.070681 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:26.070483 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:27.071233 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:27.071179 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:27.071658 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:27.071330 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:28.070803 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:28.070764 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:28.070994 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:28.070768 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:28.071046 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.070990 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:28.071046 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.070872 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:28.543483 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:28.543447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:28.543857 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.543592 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:28.543857 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.543658 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:35:44.543642626 +0000 UTC m=+34.137998123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:28.644590 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:28.644550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:28.644783 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:28.644612 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:28.644783 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644730 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:28.644783 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644733 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:28.644783 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644757 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:28.644783 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644770 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:28.644978 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644797 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:44.644779632 +0000 UTC m=+34.239135152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:28.644978 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:28.644815 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:44.644806252 +0000 UTC m=+34.239161749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:29.071183 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:29.071141 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:29.071371 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:29.071310 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:30.070552 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:30.070504 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:30.071033 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:30.070552 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:30.071033 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:30.070659 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:30.071033 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:30.070783 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:31.071147 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:31.071115 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:31.071637 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:31.071237 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:32.070915 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:32.070878 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:32.071169 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:32.070889 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:32.071169 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:32.071043 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:32.071169 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:32.071092 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:33.071207 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:33.071161 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:33.071367 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:33.071296 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:34.071557 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.071086 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:34.072418 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.071150 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:34.072418 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:34.071619 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:34.072418 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:34.071692 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:34.271248 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.271215 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="49a224a04ecc1ffae967e7aae17288e80956f204fea80779c1647050bfb337c2" exitCode=0 Jul 10 14:35:34.271395 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.271310 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"49a224a04ecc1ffae967e7aae17288e80956f204fea80779c1647050bfb337c2"} Jul 10 14:35:34.273839 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.273812 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" event={"ID":"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1","Type":"ContainerStarted","Data":"7766c710e44f3a892c19618cb37b0115e8160df01da5db35f38351a96c47b7d7"} Jul 10 14:35:34.276526 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.276497 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d28x6" event={"ID":"dc7ceef6-890f-4ff7-93ec-f2568a64d3cb","Type":"ContainerStarted","Data":"aa8b61d07b23f4ec2a9a74cd08604f1bbff0db1ac21bb02d815fc0c4f861668d"} Jul 10 14:35:34.280457 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280438 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:35:34.280784 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280758 2572 generic.go:356] "Generic (PLEG): container finished" podID="1ba93023-66ea-493f-80b8-ec6119aa9c36" containerID="ec1e2a27b496e1419f3814655f5e49d26dd4271f8bc45c4a78bb14626b1e5515" exitCode=1 Jul 10 14:35:34.280850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280819 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"e6647b07361b18c9b0db43d9bcdaa77a083021e24fbac2e944a98d49c19d5fa8"} Jul 10 14:35:34.280900 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280869 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"8c16e365156388290061ccdefcf6b238c43eaa3f449cf85e1c528810e66dfd09"} Jul 10 14:35:34.280956 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280898 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"a50aaf83027684e90410040971ced5b257e534d8b7d0322966a73d10592d5059"} Jul 10 14:35:34.280956 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280928 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"f0e2654c11698d60e78597b969ce2ef85efda45117616abf1867c0f71e021cbd"} Jul 10 14:35:34.280956 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280954 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerDied","Data":"ec1e2a27b496e1419f3814655f5e49d26dd4271f8bc45c4a78bb14626b1e5515"} Jul 10 14:35:34.281127 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.280982 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"139549fc8b68812d07b6d1e23b8fb364de4413f28360420390e9d3ed3e38b6e7"} Jul 10 14:35:34.282685 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.282663 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" event={"ID":"ba4b1c2f-f22a-4ccb-8b8c-5dae48199094","Type":"ContainerStarted","Data":"2f055204f566b786955a7febf871ff46e92d600cbf708678f62799a9a581e425"} Jul 10 14:35:34.284540 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.284517 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-txdwq" event={"ID":"b224fd08-5b50-4b09-8c9c-30e22d27b052","Type":"ContainerStarted","Data":"9de634f0d4c67b9617b7bb2fd71ab8b0f9a5a3211758c7bc7251fa7c8663a6d6"} Jul 10 14:35:34.286060 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.286026 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x55kl" event={"ID":"9cb7717f-6743-4d0d-b398-1a32ab7bd6e6","Type":"ContainerStarted","Data":"469ffde9f00c1952b7d8f47da3192005844beb4321787beca0c1108f315997d9"} Jul 10 14:35:34.287530 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.287504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r6trl" event={"ID":"57c89553-a981-4c32-8289-e82919822315","Type":"ContainerStarted","Data":"206fdc211ebf9566270a468eb90a5a3bac7e9c338ce1b0d7e8c345024716649d"} Jul 10 14:35:34.300474 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.300429 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-92.ec2.internal" podStartSLOduration=22.300414453 podStartE2EDuration="22.300414453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:16.237300013 +0000 UTC m=+5.831655534" watchObservedRunningTime="2026-07-10 14:35:34.300414453 +0000 UTC m=+23.894769971" Jul 10 14:35:34.320524 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.320482 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x55kl" podStartSLOduration=8.979270960000001 podStartE2EDuration="23.3204667s" totalImagesPullingTime="14.34119574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:34.320305992 +0000 UTC m=+23.914661510" watchObservedRunningTime="2026-07-10 14:35:34.3204667 +0000 UTC m=+23.914822219" Jul 10 14:35:34.339095 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.339040 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-txdwq" podStartSLOduration=4.644099184 podStartE2EDuration="23.339017129s" totalImagesPullingTime="18.694917945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:34.33873657 +0000 UTC m=+23.933092089" watchObservedRunningTime="2026-07-10 14:35:34.339017129 +0000 UTC m=+23.933372649" Jul 10 14:35:34.339281 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.339249 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 14:35:34.360790 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.360728 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d28x6" podStartSLOduration=4.412554457 podStartE2EDuration="23.360711371s" totalImagesPullingTime="18.948156914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:34.360037257 +0000 UTC m=+23.954392775" watchObservedRunningTime="2026-07-10 14:35:34.360711371 +0000 UTC m=+23.955066890" Jul 10 14:35:34.391057 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.391004 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r6trl" podStartSLOduration=4.574532033 podStartE2EDuration="23.390990314s" totalImagesPullingTime="18.816458281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:34.390398101 +0000 UTC m=+23.984753620" watchObservedRunningTime="2026-07-10 14:35:34.390990314 +0000 UTC m=+23.985345832" Jul 10 14:35:34.423894 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.423845 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9pz7s" podStartSLOduration=4.490998712 podStartE2EDuration="23.423829312s" totalImagesPullingTime="18.9328306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:34.423059456 +0000 UTC m=+24.017414986" watchObservedRunningTime="2026-07-10 14:35:34.423829312 +0000 UTC m=+24.018184830" Jul 10 14:35:34.573562 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.573522 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:34.574073 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.574057 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:34.877911 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.877726 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T14:35:34.339264691Z","UUID":"0183d44c-36e7-4886-a6b7-58e4f46535c9","Handler":null,"Name":"","Endpoint":""} Jul 10 14:35:34.880738 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.880705 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 14:35:34.880900 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:34.880754 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 14:35:35.074042 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.074010 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:35.074501 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:35.074187 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:35.292739 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.292702 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7rqhp" event={"ID":"0dbbca7d-f975-49ff-8726-92bcd15c02bd","Type":"ContainerStarted","Data":"af7c65fda8fe958019ea46b8dc2d2e58addf4d23fe10bc9c56cce427f4a8abdb"} Jul 10 14:35:35.295929 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.295899 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" event={"ID":"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1","Type":"ContainerStarted","Data":"5b0c5d8180284ee237730adb4a626a3da01f13d54a55767d63b0e349478aa649"} Jul 10 14:35:35.296358 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.296335 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:35.296815 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.296797 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x55kl" Jul 10 14:35:35.310807 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:35.310763 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7rqhp" podStartSLOduration=5.6121075860000005 podStartE2EDuration="24.310743838s" totalImagesPullingTime="18.698636252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:35.310239469 +0000 UTC m=+24.904594989" watchObservedRunningTime="2026-07-10 14:35:35.310743838 +0000 UTC m=+24.905099358" Jul 10 14:35:36.070984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.070948 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:36.070984 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.070976 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:36.071246 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:36.071102 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:36.071315 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:36.071285 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:36.300458 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.300373 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" event={"ID":"7fa67dcc-e1d7-40e2-95a2-ab8c2c8b10e1","Type":"ContainerStarted","Data":"d7ad8f98d6d0fd9e725d332be467bc64a8b64d2ff6b9ee11d2bd8159cdeb69db"} Jul 10 14:35:36.304460 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.304432 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:35:36.304967 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.304937 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"09b8079f38cf4ed3438ea605605587b4a7355b958059bea59f794eae8313f9dd"} Jul 10 14:35:36.323698 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:36.323650 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zbfm7" podStartSLOduration=4.594398061 podStartE2EDuration="25.323632244s" totalImagesPullingTime="20.729234183s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:36.323131682 +0000 UTC m=+25.917487192" watchObservedRunningTime="2026-07-10 14:35:36.323632244 +0000 UTC m=+25.917987801" Jul 10 14:35:37.070667 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:37.070488 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:37.070834 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:37.070742 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:38.071326 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:38.071295 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:38.071812 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:38.071299 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:38.071812 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:38.071400 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:38.071812 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:38.071509 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:39.070547 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:39.070506 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:39.070724 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:39.070669 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:40.071248 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.070989 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:40.071248 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.070989 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:40.071807 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:40.071272 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:40.071807 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:40.071331 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:40.315573 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.315536 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="20d31eabd13b9c00d9f3d2183497720e58dbc7c3a9cb84888d0f3919a5d70013" exitCode=0 Jul 10 14:35:40.315728 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.315628 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"20d31eabd13b9c00d9f3d2183497720e58dbc7c3a9cb84888d0f3919a5d70013"} Jul 10 14:35:40.319019 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.318998 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:35:40.319386 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.319364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"c9b7880b38cac0f93407c55c876d70a8b065e0e39101ce0f9c27f725c79ecf4a"} Jul 10 14:35:40.319654 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.319632 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:40.319851 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.319832 2572 scope.go:117] "RemoveContainer" containerID="ec1e2a27b496e1419f3814655f5e49d26dd4271f8bc45c4a78bb14626b1e5515" Jul 10 14:35:40.337180 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:40.337154 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:41.072135 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.072103 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:41.072646 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:41.072220 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:41.324368 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.324070 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerStarted","Data":"70bed6b77dac5c35b99e28242f5d27394c38878ddea6b21c8a21713cbc767e90"} Jul 10 14:35:41.328875 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.328857 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:35:41.329186 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.329160 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" event={"ID":"1ba93023-66ea-493f-80b8-ec6119aa9c36","Type":"ContainerStarted","Data":"4ca75bd15d97dd12d97c2fc6b96d8222d260afc737cdf7a03cff4d7ea504eb0a"} Jul 10 14:35:41.329339 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.329320 2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 14:35:41.329605 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.329587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:41.347182 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.347157 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:41.510215 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.510151 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" podStartSLOduration=11.489876444 podStartE2EDuration="30.510130548s" totalImagesPullingTime="19.020254104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:41.39176067 +0000 UTC m=+30.986116189" watchObservedRunningTime="2026-07-10 14:35:41.510130548 +0000 UTC m=+31.104486066" Jul 10 14:35:41.510548 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.510532 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lsnd7"] Jul 10 14:35:41.510681 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.510668 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:41.510790 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:41.510771 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:41.515386 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.515363 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hftj9"] Jul 10 14:35:41.515498 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.515463 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:41.515542 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:41.515530 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:41.519456 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.519402 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m4kfh"] Jul 10 14:35:41.519564 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.519519 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:41.519658 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:41.519639 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:41.675673 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:41.675635 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:35:42.333907 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:42.333829 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="70bed6b77dac5c35b99e28242f5d27394c38878ddea6b21c8a21713cbc767e90" exitCode=0 Jul 10 14:35:42.334422 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:42.333913 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"70bed6b77dac5c35b99e28242f5d27394c38878ddea6b21c8a21713cbc767e90"} Jul 10 14:35:43.073422 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:43.073395 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:43.073601 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:43.073398 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:43.073601 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:43.073506 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:43.073601 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:43.073594 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:43.073777 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:43.073398 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:43.073777 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:43.073716 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:44.341561 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:44.341525 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="5629af00acc20bb46c6c8873277885ab32d1b2fc33a73cd7ec0f73c4cf1e70a4" exitCode=0 Jul 10 14:35:44.342346 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:44.341583 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"5629af00acc20bb46c6c8873277885ab32d1b2fc33a73cd7ec0f73c4cf1e70a4"} Jul 10 14:35:44.552486 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:44.552447 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:44.552654 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.552559 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:44.552654 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.552613 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret podName:0556a41c-eefb-467e-91bb-209b5bf6d33d nodeName:}" failed. No retries permitted until 2026-07-10 14:36:16.552599359 +0000 UTC m=+66.146954856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret") pod "global-pull-secret-syncer-lsnd7" (UID: "0556a41c-eefb-467e-91bb-209b5bf6d33d") : object "kube-system"/"original-pull-secret" not registered Jul 10 14:35:44.653144 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:44.653056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:44.653144 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:44.653100 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653219 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653233 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653259 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653270 2572 projected.go:196] Error preparing data for projected volume kube-api-access-sxmvs for pod openshift-network-diagnostics/network-check-target-hftj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653284 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs podName:1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:16.653264501 +0000 UTC m=+66.247619999 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs") pod "network-metrics-daemon-m4kfh" (UID: "1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 14:35:44.653346 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:44.653308 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs podName:15c4d254-2047-4e9f-a82b-c9668c7afa90 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:16.653296367 +0000 UTC m=+66.247651864 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sxmvs" (UniqueName: "kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs") pod "network-check-target-hftj9" (UID: "15c4d254-2047-4e9f-a82b-c9668c7afa90") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 14:35:45.070815 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.070726 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:45.070962 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:45.070852 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hftj9" podUID="15c4d254-2047-4e9f-a82b-c9668c7afa90" Jul 10 14:35:45.071290 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.071267 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:45.071404 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:45.071374 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lsnd7" podUID="0556a41c-eefb-467e-91bb-209b5bf6d33d" Jul 10 14:35:45.071564 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.071458 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:45.071564 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:45.071543 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m4kfh" podUID="1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4" Jul 10 14:35:45.715887 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.715860 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-92.ec2.internal" event="NodeReady" Jul 10 14:35:45.716418 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.715989 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 14:35:45.818307 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.817610 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z4p47"] Jul 10 14:35:45.841674 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.841643 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-t6tmh"] Jul 10 14:35:45.842564 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.842540 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:45.846649 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.846622 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 14:35:45.846887 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.846870 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 14:35:45.847227 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.847173 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmfjj\"" Jul 10 14:35:45.847532 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.847510 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 14:35:45.847738 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.847721 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 14:35:45.856461 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.856440 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z4p47"] Jul 10 14:35:45.856599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.856485 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t6tmh"] Jul 10 14:35:45.856599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.856509 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q2zdk"] Jul 10 14:35:45.856819 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.856795 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:45.860738 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.860719 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 14:35:45.861831 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.861808 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 14:35:45.863463 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.863439 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lwwtb\"" Jul 10 14:35:45.877221 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.877178 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2zdk"] Jul 10 14:35:45.877366 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.877308 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:45.880327 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.880304 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-crmm7\"" Jul 10 14:35:45.880522 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.880462 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 14:35:45.880522 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.880497 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 14:35:45.880642 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.880538 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 14:35:45.964441 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964363 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-tmp-dir\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:45.964441 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964405 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:45.964441 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964435 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31f13d04-41b9-4251-bfef-d446f9e4eee8-data-volume\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:45.964699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964460 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:45.964699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964553 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a9ad40e-433f-4810-b306-802e5366572f-cert\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:45.964699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964579 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx8mz\" (UniqueName: \"kubernetes.io/projected/7a9ad40e-433f-4810-b306-802e5366572f-kube-api-access-qx8mz\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:45.964699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964598 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lcfk\" (UniqueName: \"kubernetes.io/projected/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-kube-api-access-7lcfk\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:45.964699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964641 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31f13d04-41b9-4251-bfef-d446f9e4eee8-crio-socket\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:45.964877 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964701 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-config-volume\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:45.964877 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964775 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:45.964877 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:45.964845 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhxrk\" (UniqueName: \"kubernetes.io/projected/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-api-access-dhxrk\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066129 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066096 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31f13d04-41b9-4251-bfef-d446f9e4eee8-crio-socket\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-config-volume\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.066319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066208 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhxrk\" (UniqueName: \"kubernetes.io/projected/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-api-access-dhxrk\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066287 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-tmp-dir\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.066319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066314 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066328 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066341 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31f13d04-41b9-4251-bfef-d446f9e4eee8-data-volume\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066365 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066392 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/31f13d04-41b9-4251-bfef-d446f9e4eee8-crio-socket\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066410 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.566389261 +0000 UTC m=+36.160744783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : configmap "kube-rbac-proxy" not found Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a9ad40e-433f-4810-b306-802e5366572f-cert\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066457 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx8mz\" (UniqueName: \"kubernetes.io/projected/7a9ad40e-433f-4810-b306-802e5366572f-kube-api-access-qx8mz\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lcfk\" (UniqueName: \"kubernetes.io/projected/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-kube-api-access-7lcfk\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066521 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066546 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls podName:99eeec30-4f73-403f-93e7-db1d9e4a1a8a nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.566535088 +0000 UTC m=+36.160890585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls") pod "dns-default-t6tmh" (UID: "99eeec30-4f73-403f-93e7-db1d9e4a1a8a") : secret "dns-default-metrics-tls" not found Jul 10 14:35:46.066576 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.066567 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:46.566553866 +0000 UTC m=+36.160909369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:35:46.067093 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066629 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-tmp-dir\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.067093 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066755 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/31f13d04-41b9-4251-bfef-d446f9e4eee8-data-volume\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.067093 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.066866 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-config-volume\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.071510 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.071485 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a9ad40e-433f-4810-b306-802e5366572f-cert\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:46.079054 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.079023 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx8mz\" (UniqueName: \"kubernetes.io/projected/7a9ad40e-433f-4810-b306-802e5366572f-kube-api-access-qx8mz\") pod \"ingress-canary-q2zdk\" (UID: \"7a9ad40e-433f-4810-b306-802e5366572f\") " pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:46.079545 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.079504 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lcfk\" (UniqueName: \"kubernetes.io/projected/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-kube-api-access-7lcfk\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.079909 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.079889 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhxrk\" (UniqueName: \"kubernetes.io/projected/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-api-access-dhxrk\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.187634 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.187573 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q2zdk" Jul 10 14:35:46.410958 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.410735 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q2zdk"] Jul 10 14:35:46.416139 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:46.416108 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9ad40e_433f_4810_b306_802e5366572f.slice/crio-cad0610086d5312736a43fe3722db7a76cb26819448f0f502bbccd063c5ed2c2 WatchSource:0}: Error finding container cad0610086d5312736a43fe3722db7a76cb26819448f0f502bbccd063c5ed2c2: Status 404 returned error can't find the container with id cad0610086d5312736a43fe3722db7a76cb26819448f0f502bbccd063c5ed2c2 Jul 10 14:35:46.571779 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.571665 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.571779 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.571729 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:46.571779 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.571754 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571797 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571875 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571903 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571879 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:47.571861855 +0000 UTC m=+37.166217369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : configmap "kube-rbac-proxy" not found Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571980 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:47.571963893 +0000 UTC m=+37.166319393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:35:46.572056 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:46.571998 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls podName:99eeec30-4f73-403f-93e7-db1d9e4a1a8a nodeName:}" failed. No retries permitted until 2026-07-10 14:35:47.57198673 +0000 UTC m=+37.166342233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls") pod "dns-default-t6tmh" (UID: "99eeec30-4f73-403f-93e7-db1d9e4a1a8a") : secret "dns-default-metrics-tls" not found Jul 10 14:35:46.643151 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.643114 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89"] Jul 10 14:35:46.672645 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.672596 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89"] Jul 10 14:35:46.672797 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.672734 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" Jul 10 14:35:46.676482 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.676178 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 14:35:46.676482 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.676309 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zszj2\"" Jul 10 14:35:46.676482 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.676179 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 14:35:46.699278 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.699255 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-txdwq_b224fd08-5b50-4b09-8c9c-30e22d27b052/dns-node-resolver/0.log" Jul 10 14:35:46.773304 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.773273 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9tn\" (UniqueName: \"kubernetes.io/projected/6db94d02-685e-4d88-aca9-d5e4eead3481-kube-api-access-rk9tn\") pod \"migrator-7f59b54f6c-crz89\" (UID: \"6db94d02-685e-4d88-aca9-d5e4eead3481\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" Jul 10 14:35:46.874306 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.874269 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9tn\" (UniqueName: \"kubernetes.io/projected/6db94d02-685e-4d88-aca9-d5e4eead3481-kube-api-access-rk9tn\") pod \"migrator-7f59b54f6c-crz89\" (UID: \"6db94d02-685e-4d88-aca9-d5e4eead3481\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" Jul 10 14:35:46.889275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.889246 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g5nq9"] Jul 10 14:35:46.891240 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.891186 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9tn\" (UniqueName: \"kubernetes.io/projected/6db94d02-685e-4d88-aca9-d5e4eead3481-kube-api-access-rk9tn\") pod \"migrator-7f59b54f6c-crz89\" (UID: \"6db94d02-685e-4d88-aca9-d5e4eead3481\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" Jul 10 14:35:46.904631 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.904600 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g5nq9"] Jul 10 14:35:46.904749 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.904694 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:46.908229 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.908204 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 14:35:46.908399 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.908247 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 14:35:46.908399 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.908208 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dbswd\"" Jul 10 14:35:46.908557 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.908424 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 14:35:46.908557 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.908550 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 14:35:46.975013 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.974976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-cabundle\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:46.975209 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.975066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvqk\" (UniqueName: \"kubernetes.io/projected/62f5d30e-374f-46a2-9506-5a207d2f0159-kube-api-access-7fvqk\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:46.975209 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.975131 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-key\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:46.983966 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:46.983939 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" Jul 10 14:35:47.073836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.073670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:35:47.073836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.073700 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:35:47.073836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.073669 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:35:47.075758 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.075736 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-cabundle\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.075879 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.075799 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvqk\" (UniqueName: \"kubernetes.io/projected/62f5d30e-374f-46a2-9506-5a207d2f0159-kube-api-access-7fvqk\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.075879 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.075857 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-key\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.076636 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.076616 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-cabundle\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.078877 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.078850 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/62f5d30e-374f-46a2-9506-5a207d2f0159-signing-key\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.081142 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081111 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 14:35:47.081447 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081422 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 14:35:47.081555 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081527 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8k94\"" Jul 10 14:35:47.081622 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081607 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 14:35:47.081687 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081624 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 14:35:47.081687 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.081683 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w457z\"" Jul 10 14:35:47.088996 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.088977 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvqk\" (UniqueName: \"kubernetes.io/projected/62f5d30e-374f-46a2-9506-5a207d2f0159-kube-api-access-7fvqk\") pod \"service-ca-79f68b9f4f-g5nq9\" (UID: \"62f5d30e-374f-46a2-9506-5a207d2f0159\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.164660 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.164535 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89"] Jul 10 14:35:47.170104 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:47.170065 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db94d02_685e_4d88_aca9_d5e4eead3481.slice/crio-2e98ab02e0c4b9c904d180da9f0f366bdb49e07dce8cedb4dbd91b3a228562fd WatchSource:0}: Error finding container 2e98ab02e0c4b9c904d180da9f0f366bdb49e07dce8cedb4dbd91b3a228562fd: Status 404 returned error can't find the container with id 2e98ab02e0c4b9c904d180da9f0f366bdb49e07dce8cedb4dbd91b3a228562fd Jul 10 14:35:47.213517 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.213487 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" Jul 10 14:35:47.362383 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.362285 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" event={"ID":"6db94d02-685e-4d88-aca9-d5e4eead3481","Type":"ContainerStarted","Data":"2e98ab02e0c4b9c904d180da9f0f366bdb49e07dce8cedb4dbd91b3a228562fd"} Jul 10 14:35:47.363890 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.363860 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2zdk" event={"ID":"7a9ad40e-433f-4810-b306-802e5366572f","Type":"ContainerStarted","Data":"cad0610086d5312736a43fe3722db7a76cb26819448f0f502bbccd063c5ed2c2"} Jul 10 14:35:47.405063 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.405028 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g5nq9"] Jul 10 14:35:47.409276 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:35:47.409239 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f5d30e_374f_46a2_9506_5a207d2f0159.slice/crio-caceb47c7520daf46b9a196b1ef5d070b2229118aa3fdd40189a1b24107d0e2b WatchSource:0}: Error finding container caceb47c7520daf46b9a196b1ef5d070b2229118aa3fdd40189a1b24107d0e2b: Status 404 returned error can't find the container with id caceb47c7520daf46b9a196b1ef5d070b2229118aa3fdd40189a1b24107d0e2b Jul 10 14:35:47.581887 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.581855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:47.582061 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.581910 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:47.582061 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.581988 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582056 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582071 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582076 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582136 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:49.58211801 +0000 UTC m=+39.176473524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : configmap "kube-rbac-proxy" not found Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582154 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls podName:99eeec30-4f73-403f-93e7-db1d9e4a1a8a nodeName:}" failed. No retries permitted until 2026-07-10 14:35:49.582145156 +0000 UTC m=+39.176500660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls") pod "dns-default-t6tmh" (UID: "99eeec30-4f73-403f-93e7-db1d9e4a1a8a") : secret "dns-default-metrics-tls" not found Jul 10 14:35:47.582217 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:47.582172 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:49.582163265 +0000 UTC m=+39.176518764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:35:47.852541 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:47.852466 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r6trl_57c89553-a981-4c32-8289-e82919822315/node-ca/0.log" Jul 10 14:35:48.368930 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:48.368889 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" event={"ID":"62f5d30e-374f-46a2-9506-5a207d2f0159","Type":"ContainerStarted","Data":"caceb47c7520daf46b9a196b1ef5d070b2229118aa3fdd40189a1b24107d0e2b"} Jul 10 14:35:49.601074 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:49.601027 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:49.601105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:49.601133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601175 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601264 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601279 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.601257013 +0000 UTC m=+43.195612524 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : configmap "kube-rbac-proxy" not found Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601280 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601306 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.601294451 +0000 UTC m=+43.195649948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:35:49.601552 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:49.601333 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls podName:99eeec30-4f73-403f-93e7-db1d9e4a1a8a nodeName:}" failed. No retries permitted until 2026-07-10 14:35:53.601318376 +0000 UTC m=+43.195673880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls") pod "dns-default-t6tmh" (UID: "99eeec30-4f73-403f-93e7-db1d9e4a1a8a") : secret "dns-default-metrics-tls" not found Jul 10 14:35:51.377981 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.377684 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" event={"ID":"62f5d30e-374f-46a2-9506-5a207d2f0159","Type":"ContainerStarted","Data":"f253580a871f225086c3f095498a05ba1a302c2cc89d984d1e6197a24450ad28"} Jul 10 14:35:51.379613 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.379585 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q2zdk" event={"ID":"7a9ad40e-433f-4810-b306-802e5366572f","Type":"ContainerStarted","Data":"90becbfb5452ab89ef51ed270802ee39fb847092bb69fc448b04ecd2fabc4f2c"} Jul 10 14:35:51.382558 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.382532 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="70f8fefda269dffff3954a76f254d490052e39da5e5dfc86aae8a4744eb5b95d" exitCode=0 Jul 10 14:35:51.382665 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.382607 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"70f8fefda269dffff3954a76f254d490052e39da5e5dfc86aae8a4744eb5b95d"} Jul 10 14:35:51.384481 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.384451 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" event={"ID":"6db94d02-685e-4d88-aca9-d5e4eead3481","Type":"ContainerStarted","Data":"6245dca294a05850a3cd57515394a14d8bb83afade668fadce709813a433e2c3"} Jul 10 14:35:51.384579 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.384501 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" event={"ID":"6db94d02-685e-4d88-aca9-d5e4eead3481","Type":"ContainerStarted","Data":"8727b73447eff45c39f3a367ce216324306ed6f67def18e1f7e81943b9b41606"} Jul 10 14:35:51.400785 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.400742 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-g5nq9" podStartSLOduration=1.669215163 podStartE2EDuration="5.40072551s" totalImagesPullingTime="3.731510347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:51.399787125 +0000 UTC m=+40.994142645" watchObservedRunningTime="2026-07-10 14:35:51.40072551 +0000 UTC m=+40.995081028" Jul 10 14:35:51.459526 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.459465 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-crz89" podStartSLOduration=1.525031302 podStartE2EDuration="5.459443055s" totalImagesPullingTime="3.934411753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:51.458964806 +0000 UTC m=+41.053320328" watchObservedRunningTime="2026-07-10 14:35:51.459443055 +0000 UTC m=+41.053798554" Jul 10 14:35:51.488664 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:51.488616 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q2zdk" podStartSLOduration=1.800479946 podStartE2EDuration="6.48859891s" totalImagesPullingTime="4.688118964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:51.487019204 +0000 UTC m=+41.081374722" watchObservedRunningTime="2026-07-10 14:35:51.48859891 +0000 UTC m=+41.082954429" Jul 10 14:35:52.394757 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:52.394666 2572 generic.go:356] "Generic (PLEG): container finished" podID="97c0fe49-1932-48b3-9f6a-0b4b0d0844fd" containerID="d740f7a6c7c425ddca98349c15210b440f5b7e98493a93410c0cf7124831287f" exitCode=0 Jul 10 14:35:52.395114 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:52.394768 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerDied","Data":"d740f7a6c7c425ddca98349c15210b440f5b7e98493a93410c0cf7124831287f"} Jul 10 14:35:53.400824 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:53.400788 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" event={"ID":"97c0fe49-1932-48b3-9f6a-0b4b0d0844fd","Type":"ContainerStarted","Data":"81fe1ae41430f3fcddeaec26c687fe3d29f60736083aefa1dcfa0e6a2114758f"} Jul 10 14:35:53.429731 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:53.429678 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n6hkm" podStartSLOduration=9.480766466 podStartE2EDuration="42.42966026s" totalImagesPullingTime="32.948893794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:35:53.427602236 +0000 UTC m=+43.021957767" watchObservedRunningTime="2026-07-10 14:35:53.42966026 +0000 UTC m=+43.024015778" Jul 10 14:35:53.637411 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:53.637369 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:35:53.637411 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:53.637414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:35:53.637465 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637525 2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637529 2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637567 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637579 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:01.637566307 +0000 UTC m=+51.231921805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : configmap "kube-rbac-proxy" not found Jul 10 14:35:53.637614 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637603 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls podName:99eeec30-4f73-403f-93e7-db1d9e4a1a8a nodeName:}" failed. No retries permitted until 2026-07-10 14:36:01.637584603 +0000 UTC m=+51.231940099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls") pod "dns-default-t6tmh" (UID: "99eeec30-4f73-403f-93e7-db1d9e4a1a8a") : secret "dns-default-metrics-tls" not found Jul 10 14:35:53.637806 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:35:53.637620 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:01.637612205 +0000 UTC m=+51.231967704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:36:01.703970 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.703926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:01.704453 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.704007 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:36:01.704453 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.704042 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:01.704453 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:01.704170 2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 14:36:01.704453 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:01.704264 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls podName:31f13d04-41b9-4251-bfef-d446f9e4eee8 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:17.70424594 +0000 UTC m=+67.298601443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z4p47" (UID: "31f13d04-41b9-4251-bfef-d446f9e4eee8") : secret "insights-runtime-extractor-tls" not found Jul 10 14:36:01.704627 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.704589 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/31f13d04-41b9-4251-bfef-d446f9e4eee8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:01.707436 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.707413 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/99eeec30-4f73-403f-93e7-db1d9e4a1a8a-metrics-tls\") pod \"dns-default-t6tmh\" (UID: \"99eeec30-4f73-403f-93e7-db1d9e4a1a8a\") " pod="openshift-dns/dns-default-t6tmh" Jul 10 14:36:01.768834 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.768791 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-t6tmh" Jul 10 14:36:01.928780 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:01.928747 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-t6tmh"] Jul 10 14:36:01.932161 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:01.932133 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99eeec30_4f73_403f_93e7_db1d9e4a1a8a.slice/crio-669950be622f00bb17b91994d4a0a0f865102809776404ef26f82562dcbfd530 WatchSource:0}: Error finding container 669950be622f00bb17b91994d4a0a0f865102809776404ef26f82562dcbfd530: Status 404 returned error can't find the container with id 669950be622f00bb17b91994d4a0a0f865102809776404ef26f82562dcbfd530 Jul 10 14:36:02.424399 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:02.424352 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t6tmh" event={"ID":"99eeec30-4f73-403f-93e7-db1d9e4a1a8a","Type":"ContainerStarted","Data":"669950be622f00bb17b91994d4a0a0f865102809776404ef26f82562dcbfd530"} Jul 10 14:36:05.434893 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:05.434862 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t6tmh" event={"ID":"99eeec30-4f73-403f-93e7-db1d9e4a1a8a","Type":"ContainerStarted","Data":"efe24f8648eef494420591af021817b5fa6b44aaaba26ef752f94c05c64cb18d"} Jul 10 14:36:06.439660 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:06.439627 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-t6tmh" event={"ID":"99eeec30-4f73-403f-93e7-db1d9e4a1a8a","Type":"ContainerStarted","Data":"93830487ebb20a211830d4b0477f4a1f2f1f88cc4986228b80231f74ba04d62d"} Jul 10 14:36:06.440051 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:06.439775 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-t6tmh" Jul 10 14:36:06.464474 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:06.464421 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-t6tmh" podStartSLOduration=18.174649861 podStartE2EDuration="21.464405954s" totalImagesPullingTime="3.289756093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:06.462942071 +0000 UTC m=+56.057297602" watchObservedRunningTime="2026-07-10 14:36:06.464405954 +0000 UTC m=+56.058761472" Jul 10 14:36:11.751321 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.751286 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq"] Jul 10 14:36:11.784421 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.784382 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq"] Jul 10 14:36:11.784603 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.784482 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.787611 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.787590 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 14:36:11.787744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.787631 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 14:36:11.787953 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.787931 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 14:36:11.788424 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.788407 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 14:36:11.788885 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.788871 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 14:36:11.788885 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.788882 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 14:36:11.789132 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.789113 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 14:36:11.844708 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.844677 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b595bc954-tfnt2"] Jul 10 14:36:11.870180 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.870149 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b595bc954-tfnt2"] Jul 10 14:36:11.870352 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.870265 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.873430 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.873399 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5vrwq\"" Jul 10 14:36:11.873430 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.873421 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 14:36:11.873727 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.873408 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 14:36:11.873727 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.873551 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 14:36:11.875414 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-trusted-ca\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.875502 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875486 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.875554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875520 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-certificates\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.875554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875548 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-bound-sa-token\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.875658 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875635 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv927\" (UniqueName: \"kubernetes.io/projected/691b9224-8a4a-47e0-9ced-1ff7681005b2-kube-api-access-mv927\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.875706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875681 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.875752 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875708 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.875875 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875772 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cea3443d-16f0-4e68-86f8-ad49b705d64b-ca-trust-extracted\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.875949 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875894 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k266g\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-kube-api-access-k266g\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.876001 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.875968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-installation-pull-secrets\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.876046 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.876000 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/691b9224-8a4a-47e0-9ced-1ff7681005b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.876046 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.876028 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.876123 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.876057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-tls\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.876173 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.876142 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-image-registry-private-configuration\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.880410 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.880392 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 14:36:11.976634 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976598 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-image-registry-private-configuration\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976634 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976632 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-trusted-ca\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976670 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976689 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-certificates\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976707 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-bound-sa-token\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv927\" (UniqueName: \"kubernetes.io/projected/691b9224-8a4a-47e0-9ced-1ff7681005b2-kube-api-access-mv927\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976764 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976784 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976805 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cea3443d-16f0-4e68-86f8-ad49b705d64b-ca-trust-extracted\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976828 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k266g\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-kube-api-access-k266g\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.976886 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976875 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-installation-pull-secrets\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.977360 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976904 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/691b9224-8a4a-47e0-9ced-1ff7681005b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.977360 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976930 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.977360 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.976955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-tls\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.979233 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.977802 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-certificates\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.979233 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.978687 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cea3443d-16f0-4e68-86f8-ad49b705d64b-ca-trust-extracted\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.979233 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.978910 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cea3443d-16f0-4e68-86f8-ad49b705d64b-trusted-ca\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.979574 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.979547 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/691b9224-8a4a-47e0-9ced-1ff7681005b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.980037 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980012 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-ca\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.980142 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980040 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-registry-tls\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.980327 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980303 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-image-registry-private-configuration\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.980376 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980351 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cea3443d-16f0-4e68-86f8-ad49b705d64b-installation-pull-secrets\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.980536 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980511 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.980859 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.980844 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.981582 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.981561 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/691b9224-8a4a-47e0-9ced-1ff7681005b2-hub\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.988906 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.988884 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv927\" (UniqueName: \"kubernetes.io/projected/691b9224-8a4a-47e0-9ced-1ff7681005b2-kube-api-access-mv927\") pod \"cluster-proxy-proxy-agent-5774f784dd-xp5nq\" (UID: \"691b9224-8a4a-47e0-9ced-1ff7681005b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:11.992446 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.992409 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-bound-sa-token\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:11.992887 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:11.992867 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k266g\" (UniqueName: \"kubernetes.io/projected/cea3443d-16f0-4e68-86f8-ad49b705d64b-kube-api-access-k266g\") pod \"image-registry-b595bc954-tfnt2\" (UID: \"cea3443d-16f0-4e68-86f8-ad49b705d64b\") " pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:12.113436 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.113400 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" Jul 10 14:36:12.179326 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.179296 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:12.311482 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.311315 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq"] Jul 10 14:36:12.315670 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:12.315639 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod691b9224_8a4a_47e0_9ced_1ff7681005b2.slice/crio-77f7e53b8f6699b32f92b3ab281ea9741b2d95d673fdce5e97ddd12d92f511c7 WatchSource:0}: Error finding container 77f7e53b8f6699b32f92b3ab281ea9741b2d95d673fdce5e97ddd12d92f511c7: Status 404 returned error can't find the container with id 77f7e53b8f6699b32f92b3ab281ea9741b2d95d673fdce5e97ddd12d92f511c7 Jul 10 14:36:12.348416 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.348324 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b595bc954-tfnt2"] Jul 10 14:36:12.352332 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:12.352308 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea3443d_16f0_4e68_86f8_ad49b705d64b.slice/crio-03fecde01e48238bc09689c25caf10f8056f8b7b184aafcdb05e849d484c605d WatchSource:0}: Error finding container 03fecde01e48238bc09689c25caf10f8056f8b7b184aafcdb05e849d484c605d: Status 404 returned error can't find the container with id 03fecde01e48238bc09689c25caf10f8056f8b7b184aafcdb05e849d484c605d Jul 10 14:36:12.458361 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.458333 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" event={"ID":"cea3443d-16f0-4e68-86f8-ad49b705d64b","Type":"ContainerStarted","Data":"2ec2f151fe2dfdc0c5b73f270e2cfd1275b88e4c916f48bf3a095e7dfc7d2bb4"} Jul 10 14:36:12.458515 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.458381 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" event={"ID":"cea3443d-16f0-4e68-86f8-ad49b705d64b","Type":"ContainerStarted","Data":"03fecde01e48238bc09689c25caf10f8056f8b7b184aafcdb05e849d484c605d"} Jul 10 14:36:12.458515 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.458436 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:12.459736 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.459677 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" event={"ID":"691b9224-8a4a-47e0-9ced-1ff7681005b2","Type":"ContainerStarted","Data":"77f7e53b8f6699b32f92b3ab281ea9741b2d95d673fdce5e97ddd12d92f511c7"} Jul 10 14:36:12.549120 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:12.549061 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" podStartSLOduration=1.5490446530000002 podStartE2EDuration="1.549044653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:12.545472036 +0000 UTC m=+62.139827556" watchObservedRunningTime="2026-07-10 14:36:12.549044653 +0000 UTC m=+62.143400163" Jul 10 14:36:13.355595 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:13.355566 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6ccb" Jul 10 14:36:16.445266 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.445165 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-t6tmh" Jul 10 14:36:16.478228 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.478169 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" event={"ID":"691b9224-8a4a-47e0-9ced-1ff7681005b2","Type":"ContainerStarted","Data":"d57049ce09846b37ee842422bda65c07736bb7423b822d522766797c4bf783f2"} Jul 10 14:36:16.478419 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.478238 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" event={"ID":"691b9224-8a4a-47e0-9ced-1ff7681005b2","Type":"ContainerStarted","Data":"1e5d415b14a0ee7130eaa33bd2b38fede31a80f80dde7767706306b093a5fd89"} Jul 10 14:36:16.478419 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.478259 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" event={"ID":"691b9224-8a4a-47e0-9ced-1ff7681005b2","Type":"ContainerStarted","Data":"4aff89aa7cd8b859386c7230fe2a1346216319d94a252b3d1b6dab10dbff1f02"} Jul 10 14:36:16.504311 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.504253 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5774f784dd-xp5nq" podStartSLOduration=1.73457199 podStartE2EDuration="5.504230302s" totalImagesPullingTime="3.769658312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:16.50354447 +0000 UTC m=+66.097899990" watchObservedRunningTime="2026-07-10 14:36:16.504230302 +0000 UTC m=+66.098585801" Jul 10 14:36:16.616683 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.616629 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:36:16.619875 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.619856 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 14:36:16.630236 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.630209 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0556a41c-eefb-467e-91bb-209b5bf6d33d-original-pull-secret\") pod \"global-pull-secret-syncer-lsnd7\" (UID: \"0556a41c-eefb-467e-91bb-209b5bf6d33d\") " pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:36:16.717718 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.717627 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:36:16.717718 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.717682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:36:16.720943 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.720925 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 14:36:16.721013 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.720925 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 14:36:16.730831 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.730799 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4-metrics-certs\") pod \"network-metrics-daemon-m4kfh\" (UID: \"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4\") " pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:36:16.731467 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.731448 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 14:36:16.741649 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.741620 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmvs\" (UniqueName: \"kubernetes.io/projected/15c4d254-2047-4e9f-a82b-c9668c7afa90-kube-api-access-sxmvs\") pod \"network-check-target-hftj9\" (UID: \"15c4d254-2047-4e9f-a82b-c9668c7afa90\") " pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:36:16.803540 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.803512 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w457z\"" Jul 10 14:36:16.809620 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.809603 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lsnd7" Jul 10 14:36:16.810669 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.810653 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:36:16.819532 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.819511 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8k94\"" Jul 10 14:36:16.826429 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.826408 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m4kfh" Jul 10 14:36:16.958182 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.958133 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lsnd7"] Jul 10 14:36:16.962514 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:16.962481 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0556a41c_eefb_467e_91bb_209b5bf6d33d.slice/crio-790707319fa545bf4f83f6987b8bc912bdc0cb2440222cd649185cc45900c7f9 WatchSource:0}: Error finding container 790707319fa545bf4f83f6987b8bc912bdc0cb2440222cd649185cc45900c7f9: Status 404 returned error can't find the container with id 790707319fa545bf4f83f6987b8bc912bdc0cb2440222cd649185cc45900c7f9 Jul 10 14:36:16.979605 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.979534 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hftj9"] Jul 10 14:36:16.982783 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:16.982754 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c4d254_2047_4e9f_a82b_c9668c7afa90.slice/crio-dc6aae94cb21a61d4c6018ca4001f4ba7bb367510b5d35f28db1efcc870b4ce7 WatchSource:0}: Error finding container dc6aae94cb21a61d4c6018ca4001f4ba7bb367510b5d35f28db1efcc870b4ce7: Status 404 returned error can't find the container with id dc6aae94cb21a61d4c6018ca4001f4ba7bb367510b5d35f28db1efcc870b4ce7 Jul 10 14:36:16.996851 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:16.996820 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m4kfh"] Jul 10 14:36:16.999863 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:16.999843 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ee11e9a_a2e0_40b2_8868_3f5c6668e9a4.slice/crio-fadf561448cf9c9049668bbf1bef7138d2cdae8275685139a9480eab64cb8c71 WatchSource:0}: Error finding container fadf561448cf9c9049668bbf1bef7138d2cdae8275685139a9480eab64cb8c71: Status 404 returned error can't find the container with id fadf561448cf9c9049668bbf1bef7138d2cdae8275685139a9480eab64cb8c71 Jul 10 14:36:17.483213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.483158 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hftj9" event={"ID":"15c4d254-2047-4e9f-a82b-c9668c7afa90","Type":"ContainerStarted","Data":"dc6aae94cb21a61d4c6018ca4001f4ba7bb367510b5d35f28db1efcc870b4ce7"} Jul 10 14:36:17.484739 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.484706 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lsnd7" event={"ID":"0556a41c-eefb-467e-91bb-209b5bf6d33d","Type":"ContainerStarted","Data":"790707319fa545bf4f83f6987b8bc912bdc0cb2440222cd649185cc45900c7f9"} Jul 10 14:36:17.486070 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.486044 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m4kfh" event={"ID":"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4","Type":"ContainerStarted","Data":"fadf561448cf9c9049668bbf1bef7138d2cdae8275685139a9480eab64cb8c71"} Jul 10 14:36:17.731022 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.730974 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:17.734698 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.734600 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/31f13d04-41b9-4251-bfef-d446f9e4eee8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z4p47\" (UID: \"31f13d04-41b9-4251-bfef-d446f9e4eee8\") " pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:17.958051 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.958013 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmfjj\"" Jul 10 14:36:17.965905 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:17.965876 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z4p47" Jul 10 14:36:18.391975 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:18.391910 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z4p47"] Jul 10 14:36:18.397368 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:18.397180 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f13d04_41b9_4251_bfef_d446f9e4eee8.slice/crio-997a4ccacb376869fa6f61867266bd2ca62ee05fbaeb36d2a38e1832ffc9ac02 WatchSource:0}: Error finding container 997a4ccacb376869fa6f61867266bd2ca62ee05fbaeb36d2a38e1832ffc9ac02: Status 404 returned error can't find the container with id 997a4ccacb376869fa6f61867266bd2ca62ee05fbaeb36d2a38e1832ffc9ac02 Jul 10 14:36:18.492876 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:18.492788 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m4kfh" event={"ID":"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4","Type":"ContainerStarted","Data":"47743945b0bdc57e1856ea031c51a1f5ed70039963380c481ce1a264ce5c052e"} Jul 10 14:36:18.494603 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:18.494524 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4p47" event={"ID":"31f13d04-41b9-4251-bfef-d446f9e4eee8","Type":"ContainerStarted","Data":"997a4ccacb376869fa6f61867266bd2ca62ee05fbaeb36d2a38e1832ffc9ac02"} Jul 10 14:36:19.508154 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:19.508114 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m4kfh" event={"ID":"1ee11e9a-a2e0-40b2-8868-3f5c6668e9a4","Type":"ContainerStarted","Data":"7c9e7703e8cd376efdef023cadce5b52fee88c91f3d0159a428997f131ae13d0"} Jul 10 14:36:25.533180 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.533138 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lsnd7" event={"ID":"0556a41c-eefb-467e-91bb-209b5bf6d33d","Type":"ContainerStarted","Data":"dc486f39d147ac56ca15497461b3e1f562ac67fd050009a36493355e9508e6b8"} Jul 10 14:36:25.534731 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.534708 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hftj9" event={"ID":"15c4d254-2047-4e9f-a82b-c9668c7afa90","Type":"ContainerStarted","Data":"9139ae0e607e509640d04f61e2ab041a861f27629ac6afdbef55a2479b122972"} Jul 10 14:36:25.534835 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.534773 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:36:25.536343 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.536322 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4p47" event={"ID":"31f13d04-41b9-4251-bfef-d446f9e4eee8","Type":"ContainerStarted","Data":"2d7d5da0e3b4bc546104d8f26b5e466137e510650d3b116b421e6ada56cf18a3"} Jul 10 14:36:25.536445 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.536365 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4p47" event={"ID":"31f13d04-41b9-4251-bfef-d446f9e4eee8","Type":"ContainerStarted","Data":"87d46272e3820f29e1238129fdc8c26b83fedce35cef097b2ed108e4c8552f2d"} Jul 10 14:36:25.556280 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.556231 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m4kfh" podStartSLOduration=73.299643405 podStartE2EDuration="1m14.556215808s" totalImagesPullingTime="1.256572403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:19.529404053 +0000 UTC m=+69.123759574" watchObservedRunningTime="2026-07-10 14:36:25.556215808 +0000 UTC m=+75.150571317" Jul 10 14:36:25.556458 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.556376 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lsnd7" podStartSLOduration=66.891943554 podStartE2EDuration="1m14.556369942s" totalImagesPullingTime="7.664426388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:25.555282408 +0000 UTC m=+75.149637930" watchObservedRunningTime="2026-07-10 14:36:25.556369942 +0000 UTC m=+75.150725462" Jul 10 14:36:25.580739 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:25.580680 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hftj9" podStartSLOduration=66.937879461 podStartE2EDuration="1m14.580661352s" totalImagesPullingTime="7.642781891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:35:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:25.579975078 +0000 UTC m=+75.174330598" watchObservedRunningTime="2026-07-10 14:36:25.580661352 +0000 UTC m=+75.175016870" Jul 10 14:36:27.182554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.181260 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nm6rr"] Jul 10 14:36:27.187379 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.186447 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.212526 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.212494 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.215865 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"node-exporter-nm6rr\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" podUID="6317b6ca-23ee-4454-ba74-cef7e7894d32" pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.215951 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.215986 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.215986 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-sbjmx\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sbjmx\"" type="*v1.Secret" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.216010 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-client-ca\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.216064 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" Jul 10 14:36:27.216109 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:27.216075 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314252 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314282 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-wtmp\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314302 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314331 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klkdq\" (UniqueName: \"kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314355 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-tls\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314377 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-textfile\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314399 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-sys\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.316213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.314426 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-root\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415399 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415363 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-sys\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415414 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-root\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415449 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415475 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-sys\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415483 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415527 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-wtmp\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415544 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-root\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415553 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415599 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415596 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klkdq\" (UniqueName: \"kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415986 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415633 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-tls\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415986 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415665 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-textfile\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415986 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415767 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-wtmp\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:27.415986 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:27.415949 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-textfile\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.194440 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.194407 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sbjmx\"" Jul 10 14:36:28.332095 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.332063 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 14:36:28.338740 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.338713 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-tls\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.339361 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.339343 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 14:36:28.416653 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416620 2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition Jul 10 14:36:28.416832 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416690 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config podName:6317b6ca-23ee-4454-ba74-cef7e7894d32 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:28.916674032 +0000 UTC m=+78.511029529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config") pod "node-exporter-nm6rr" (UID: "6317b6ca-23ee-4454-ba74-cef7e7894d32") : failed to sync secret cache: timed out waiting for the condition Jul 10 14:36:28.416832 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416614 2572 configmap.go:193] Couldn't get configMap openshift-monitoring/metrics-client-ca: failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.416832 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416778 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca podName:6317b6ca-23ee-4454-ba74-cef7e7894d32 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:28.916757166 +0000 UTC m=+78.511112668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-client-ca" (UniqueName: "kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca") pod "node-exporter-nm6rr" (UID: "6317b6ca-23ee-4454-ba74-cef7e7894d32") : failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.416832 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416630 2572 configmap.go:193] Couldn't get configMap openshift-monitoring/node-exporter-accelerators-collector-config: failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.417000 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.416851 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config podName:6317b6ca-23ee-4454-ba74-cef7e7894d32 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:28.91683559 +0000 UTC m=+78.511191087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-accelerators-collector-config" (UniqueName: "kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config") pod "node-exporter-nm6rr" (UID: "6317b6ca-23ee-4454-ba74-cef7e7894d32") : failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.422483 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.422460 2572 projected.go:291] Couldn't get configMap openshift-monitoring/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.422551 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.422486 2572 projected.go:196] Error preparing data for projected volume kube-api-access-klkdq for pod openshift-monitoring/node-exporter-nm6rr: failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.422551 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:36:28.422519 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq podName:6317b6ca-23ee-4454-ba74-cef7e7894d32 nodeName:}" failed. No retries permitted until 2026-07-10 14:36:28.922510495 +0000 UTC m=+78.516865992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-klkdq" (UniqueName: "kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq") pod "node-exporter-nm6rr" (UID: "6317b6ca-23ee-4454-ba74-cef7e7894d32") : failed to sync configmap cache: timed out waiting for the condition Jul 10 14:36:28.454361 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.454331 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 14:36:28.552715 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.552649 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z4p47" event={"ID":"31f13d04-41b9-4251-bfef-d446f9e4eee8","Type":"ContainerStarted","Data":"7eff1b0669b5c3d55aaf059457262607dc1667401fa046aeb50e1453edfa8167"} Jul 10 14:36:28.600659 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.600556 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z4p47" podStartSLOduration=34.124900955 podStartE2EDuration="43.600536302s" totalImagesPullingTime="9.475635347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 14:35:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:28.598866603 +0000 UTC m=+78.193222124" watchObservedRunningTime="2026-07-10 14:36:28.600536302 +0000 UTC m=+78.194891821" Jul 10 14:36:28.621359 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.621328 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 14:36:28.639988 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.639962 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 14:36:28.647805 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.647782 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 14:36:28.927740 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.927649 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.927740 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.927685 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.927740 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.927704 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.928020 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.927837 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klkdq\" (UniqueName: \"kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.928274 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.928248 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-metrics-client-ca\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.928396 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.928252 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-accelerators-collector-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.930148 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.930125 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6317b6ca-23ee-4454-ba74-cef7e7894d32-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:28.930412 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:28.930390 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klkdq\" (UniqueName: \"kubernetes.io/projected/6317b6ca-23ee-4454-ba74-cef7e7894d32-kube-api-access-klkdq\") pod \"node-exporter-nm6rr\" (UID: \"6317b6ca-23ee-4454-ba74-cef7e7894d32\") " pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:29.005552 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:29.005513 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nm6rr" Jul 10 14:36:29.014862 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:29.014823 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6317b6ca_23ee_4454_ba74_cef7e7894d32.slice/crio-b99b2c8da90fca03bb5e878e00cd0e225df251c4f227c5911d493e75b7676ec9 WatchSource:0}: Error finding container b99b2c8da90fca03bb5e878e00cd0e225df251c4f227c5911d493e75b7676ec9: Status 404 returned error can't find the container with id b99b2c8da90fca03bb5e878e00cd0e225df251c4f227c5911d493e75b7676ec9 Jul 10 14:36:29.558087 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:29.558040 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nm6rr" event={"ID":"6317b6ca-23ee-4454-ba74-cef7e7894d32","Type":"ContainerStarted","Data":"b99b2c8da90fca03bb5e878e00cd0e225df251c4f227c5911d493e75b7676ec9"} Jul 10 14:36:30.563011 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:30.562977 2572 generic.go:356] "Generic (PLEG): container finished" podID="6317b6ca-23ee-4454-ba74-cef7e7894d32" containerID="97bb96ed95d5f9093d4db86e1dd80f24b3a3423204a643b8fce62f0617b4938b" exitCode=0 Jul 10 14:36:30.563453 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:30.563030 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nm6rr" event={"ID":"6317b6ca-23ee-4454-ba74-cef7e7894d32","Type":"ContainerDied","Data":"97bb96ed95d5f9093d4db86e1dd80f24b3a3423204a643b8fce62f0617b4938b"} Jul 10 14:36:31.568953 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:31.568912 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nm6rr" event={"ID":"6317b6ca-23ee-4454-ba74-cef7e7894d32","Type":"ContainerStarted","Data":"f4a8af53148d4f0c50ebf8366b3ebf814969eea049a401b531861e2f725d4c99"} Jul 10 14:36:31.569546 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:31.568965 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nm6rr" event={"ID":"6317b6ca-23ee-4454-ba74-cef7e7894d32","Type":"ContainerStarted","Data":"9e6ac5a5f18b01588ae1976fbcd71cf0acf7e540684e498a09362f23e9b4aac7"} Jul 10 14:36:31.598611 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:31.598559 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nm6rr" podStartSLOduration=3.733352548 podStartE2EDuration="4.59854222s" totalImagesPullingTime="791.652713ms" totalInitContainerRuntime="73.536959ms" isStatefulPod=true podCreationTimestamp="2026-07-10 14:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:31.597818078 +0000 UTC m=+81.192173596" watchObservedRunningTime="2026-07-10 14:36:31.59854222 +0000 UTC m=+81.192897738" Jul 10 14:36:32.480487 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.480449 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7947b8dbd-6mswc"] Jul 10 14:36:32.482542 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.482527 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.486344 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.486320 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 14:36:32.486462 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.486367 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 14:36:32.486462 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.486412 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 14:36:32.486761 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.486747 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 14:36:32.486839 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.486827 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ssdxr\"" Jul 10 14:36:32.487343 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.487327 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 14:36:32.510781 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.510745 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 14:36:32.556589 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556554 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556589 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556593 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556811 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556622 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-metrics-client-ca\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556811 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556684 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-serving-certs-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556811 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556752 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-federate-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556811 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556811 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556795 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6p6b\" (UniqueName: \"kubernetes.io/projected/9888294c-5d6b-4dfc-8394-5cf0866d44fc-kube-api-access-f6p6b\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.556971 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.556819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.559887 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.559853 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7947b8dbd-6mswc"] Jul 10 14:36:32.657657 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.657619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.657657 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.657660 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.658143 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.657682 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-metrics-client-ca\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.658143 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.657697 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-serving-certs-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658615 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-federate-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658717 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658765 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-metrics-client-ca\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6p6b\" (UniqueName: \"kubernetes.io/projected/9888294c-5d6b-4dfc-8394-5cf0866d44fc-kube-api-access-f6p6b\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658836 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-serving-certs-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.660744 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.658970 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.661186 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.661166 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.661388 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.661364 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.661501 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.661468 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-secret-telemeter-client\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.664938 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.664914 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-federate-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.665874 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.665854 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9888294c-5d6b-4dfc-8394-5cf0866d44fc-telemeter-client-tls\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.670317 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.670296 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6p6b\" (UniqueName: \"kubernetes.io/projected/9888294c-5d6b-4dfc-8394-5cf0866d44fc-kube-api-access-f6p6b\") pod \"telemeter-client-7947b8dbd-6mswc\" (UID: \"9888294c-5d6b-4dfc-8394-5cf0866d44fc\") " pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.791637 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.791554 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" Jul 10 14:36:32.939397 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:32.939364 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7947b8dbd-6mswc"] Jul 10 14:36:32.942621 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:36:32.942578 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9888294c_5d6b_4dfc_8394_5cf0866d44fc.slice/crio-6d741f2f1955b46f7307d56ee6a9419505dcd52a73c00ca573a301c4cd9ecd02 WatchSource:0}: Error finding container 6d741f2f1955b46f7307d56ee6a9419505dcd52a73c00ca573a301c4cd9ecd02: Status 404 returned error can't find the container with id 6d741f2f1955b46f7307d56ee6a9419505dcd52a73c00ca573a301c4cd9ecd02 Jul 10 14:36:33.467787 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:33.467761 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b595bc954-tfnt2" Jul 10 14:36:33.577240 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:33.577208 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" event={"ID":"9888294c-5d6b-4dfc-8394-5cf0866d44fc","Type":"ContainerStarted","Data":"6d741f2f1955b46f7307d56ee6a9419505dcd52a73c00ca573a301c4cd9ecd02"} Jul 10 14:36:35.587506 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:35.587459 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" event={"ID":"9888294c-5d6b-4dfc-8394-5cf0866d44fc","Type":"ContainerStarted","Data":"92194f00f26bd02294e1caa5b7045ceca5db6dce0db77c449b747c4d5e687273"} Jul 10 14:36:36.592800 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:36.592762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" event={"ID":"9888294c-5d6b-4dfc-8394-5cf0866d44fc","Type":"ContainerStarted","Data":"a01cc21f647a9c90ae9d853eebcad854327560c1617ccc1cab20f5c46c691be9"} Jul 10 14:36:36.593177 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:36.592820 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" event={"ID":"9888294c-5d6b-4dfc-8394-5cf0866d44fc","Type":"ContainerStarted","Data":"9446cc4a71ece46aa133730919083eea6c1089d6b1afd8d494e5ceae5322060c"} Jul 10 14:36:56.546520 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:56.546492 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hftj9" Jul 10 14:36:56.572728 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:36:56.572672 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7947b8dbd-6mswc" podStartSLOduration=21.647467159 podStartE2EDuration="24.572653947s" totalImagesPullingTime="2.925186788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:36:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:36:36.630726354 +0000 UTC m=+86.225081873" watchObservedRunningTime="2026-07-10 14:36:56.572653947 +0000 UTC m=+106.167009465" Jul 10 14:39:56.694442 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.694403 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-knj9p"] Jul 10 14:39:56.697598 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.697581 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.700683 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.700662 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 14:39:56.701753 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.701727 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vdk2w\"" Jul 10 14:39:56.701865 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.701787 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 14:39:56.710471 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.710448 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-knj9p"] Jul 10 14:39:56.854805 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.854766 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.854991 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.854822 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmfkc\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-kube-api-access-tmfkc\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.956214 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.956080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.956214 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.956177 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmfkc\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-kube-api-access-tmfkc\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.978256 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.978224 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmfkc\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-kube-api-access-tmfkc\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:56.978489 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:56.978470 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/264c3594-2637-45db-a6b1-44be61d725fe-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-knj9p\" (UID: \"264c3594-2637-45db-a6b1-44be61d725fe\") " pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:57.006582 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:57.006540 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:39:57.142072 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:57.142038 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-knj9p"] Jul 10 14:39:57.268691 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:39:57.268599 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" event={"ID":"264c3594-2637-45db-a6b1-44be61d725fe","Type":"ContainerStarted","Data":"c94e7d3b515b772aa23e3904354645bd549eef2817f230de023d9e51c82055a5"} Jul 10 14:40:00.282382 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:00.282298 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" event={"ID":"264c3594-2637-45db-a6b1-44be61d725fe","Type":"ContainerStarted","Data":"26715bee8eaf3ae582d69295b18e06f61aaa02ca9f8ffebf55e5816917ceded5"} Jul 10 14:40:00.282735 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:00.282435 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:40:00.302930 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:00.302877 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" podStartSLOduration=1.419848172 podStartE2EDuration="4.302861451s" totalImagesPullingTime="2.883013279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:00.301244861 +0000 UTC m=+289.895600378" watchObservedRunningTime="2026-07-10 14:40:00.302861451 +0000 UTC m=+289.897216969" Jul 10 14:40:06.287386 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:06.287350 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-knj9p" Jul 10 14:40:10.804557 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:10.804533 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:40:10.806148 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:10.806124 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:40:10.808715 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:10.808688 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 14:40:23.785442 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.785405 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc"] Jul 10 14:40:23.788941 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.788925 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.792722 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.792702 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rpbct\"" Jul 10 14:40:23.793092 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.793076 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 14:40:23.793405 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.793389 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 14:40:23.793535 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.793517 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 14:40:23.794314 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.794300 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 14:40:23.824220 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.824171 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc"] Jul 10 14:40:23.852683 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.852654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.852850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.852736 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.852850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.852764 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdrjs\" (UniqueName: \"kubernetes.io/projected/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-kube-api-access-pdrjs\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.852850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.852797 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.852850 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.852820 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.953890 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.953849 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.953890 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.953886 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdrjs\" (UniqueName: \"kubernetes.io/projected/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-kube-api-access-pdrjs\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.954130 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.953955 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.954130 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.953985 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.954130 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.954013 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.954412 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.954393 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-manifests\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.956699 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.956675 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-tmp\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.956802 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.956722 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-webhook-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.956802 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.956746 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:23.978068 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:23.978039 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdrjs\" (UniqueName: \"kubernetes.io/projected/3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296-kube-api-access-pdrjs\") pod \"opendatahub-operator-controller-manager-85d85459f4-t4knc\" (UID: \"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296\") " pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:24.099364 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:24.099327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:24.287818 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:24.287781 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc"] Jul 10 14:40:24.291927 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:40:24.291898 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9d95ae_01ec_4cd4_b4cb_4dacd13b0296.slice/crio-67587ed87528a75c8796518dac8c4e2872b2b1cb89787639436e4e4ee81abf75 WatchSource:0}: Error finding container 67587ed87528a75c8796518dac8c4e2872b2b1cb89787639436e4e4ee81abf75: Status 404 returned error can't find the container with id 67587ed87528a75c8796518dac8c4e2872b2b1cb89787639436e4e4ee81abf75 Jul 10 14:40:24.293752 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:24.293730 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:40:24.363603 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:24.363523 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" event={"ID":"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296","Type":"ContainerStarted","Data":"67587ed87528a75c8796518dac8c4e2872b2b1cb89787639436e4e4ee81abf75"} Jul 10 14:40:27.377970 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:27.377925 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" event={"ID":"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296","Type":"ContainerStarted","Data":"973a30b1436e44571427429c7825ccf5733363e9824490b5d36b4ca279e58110"} Jul 10 14:40:29.386674 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:29.386637 2572 generic.go:356] "Generic (PLEG): container finished" podID="3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296" containerID="973a30b1436e44571427429c7825ccf5733363e9824490b5d36b4ca279e58110" exitCode=0 Jul 10 14:40:29.387062 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:29.386705 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" event={"ID":"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296","Type":"ContainerDied","Data":"973a30b1436e44571427429c7825ccf5733363e9824490b5d36b4ca279e58110"} Jul 10 14:40:30.391765 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:30.391731 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" event={"ID":"3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296","Type":"ContainerStarted","Data":"990ab0a9ddc7ab6de4eadac48847ba81505885e2a2b6bdfa9e33df15ab5fc0c6"} Jul 10 14:40:30.392149 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:30.391946 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:30.427748 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:30.427695 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" podStartSLOduration=2.89540336 podStartE2EDuration="7.427678603s" totalImagesPullingTime="3.463566122s" totalInitContainerRuntime="1.068709121s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:30.427403052 +0000 UTC m=+320.021758592" watchObservedRunningTime="2026-07-10 14:40:30.427678603 +0000 UTC m=+320.022034514" Jul 10 14:40:41.397836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:41.397802 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85d85459f4-t4knc" Jul 10 14:40:43.371522 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.371487 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c"] Jul 10 14:40:43.378356 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.378337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.383372 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.383344 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 14:40:43.383567 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.383469 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 14:40:43.385400 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.385360 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d5bj6\"" Jul 10 14:40:43.385569 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.385441 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 14:40:43.385569 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.385441 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 14:40:43.385753 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.385692 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 14:40:43.387104 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.387083 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c"] Jul 10 14:40:43.507051 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.507014 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32145e5f-b3cc-40b9-b055-684bee700c51-manager-config\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.507051 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.507047 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.507313 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.507066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmpmf\" (UniqueName: \"kubernetes.io/projected/32145e5f-b3cc-40b9-b055-684bee700c51-kube-api-access-fmpmf\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.507313 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.507123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-metrics-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.607897 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.607860 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32145e5f-b3cc-40b9-b055-684bee700c51-manager-config\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.607897 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.607899 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.608131 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.607918 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmpmf\" (UniqueName: \"kubernetes.io/projected/32145e5f-b3cc-40b9-b055-684bee700c51-kube-api-access-fmpmf\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.608131 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.607961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-metrics-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.608685 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.608656 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/32145e5f-b3cc-40b9-b055-684bee700c51-manager-config\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.610580 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.610556 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.610712 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.610653 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/32145e5f-b3cc-40b9-b055-684bee700c51-metrics-cert\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.642477 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.642400 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmpmf\" (UniqueName: \"kubernetes.io/projected/32145e5f-b3cc-40b9-b055-684bee700c51-kube-api-access-fmpmf\") pod \"lws-controller-manager-869f49798c-vvj7c\" (UID: \"32145e5f-b3cc-40b9-b055-684bee700c51\") " pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.688020 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.687981 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:43.854514 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.854491 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c"] Jul 10 14:40:43.856701 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:40:43.856677 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32145e5f_b3cc_40b9_b055_684bee700c51.slice/crio-74a279b3afd96f2f65f432fabeb60b92ede4caaf18ebeb5e5c77e05ef4cf0ee1 WatchSource:0}: Error finding container 74a279b3afd96f2f65f432fabeb60b92ede4caaf18ebeb5e5c77e05ef4cf0ee1: Status 404 returned error can't find the container with id 74a279b3afd96f2f65f432fabeb60b92ede4caaf18ebeb5e5c77e05ef4cf0ee1 Jul 10 14:40:43.908088 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.908008 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-hvgp4"] Jul 10 14:40:43.912251 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.912230 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:43.917094 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.917074 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 14:40:43.918660 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.918641 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ghdzt\"" Jul 10 14:40:43.919558 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.919542 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 14:40:43.920108 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.920091 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 14:40:43.920389 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.920369 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 14:40:43.933545 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:43.933514 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-hvgp4"] Jul 10 14:40:44.011217 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.011155 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.011405 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.011289 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a80c0148-7b31-468f-a949-6851c247636f-tmp\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.011405 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.011324 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28tc\" (UniqueName: \"kubernetes.io/projected/a80c0148-7b31-468f-a949-6851c247636f-kube-api-access-w28tc\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.112509 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.112475 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.112706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.112532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a80c0148-7b31-468f-a949-6851c247636f-tmp\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.112706 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.112550 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28tc\" (UniqueName: \"kubernetes.io/projected/a80c0148-7b31-468f-a949-6851c247636f-kube-api-access-w28tc\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.112706 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:40:44.112642 2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 14:40:44.112889 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:40:44.112728 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs podName:a80c0148-7b31-468f-a949-6851c247636f nodeName:}" failed. No retries permitted until 2026-07-10 14:40:44.61270693 +0000 UTC m=+334.207062428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs") pod "kube-auth-proxy-74544d9668-hvgp4" (UID: "a80c0148-7b31-468f-a949-6851c247636f") : secret "kube-auth-proxy-tls" not found Jul 10 14:40:44.114817 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.114799 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a80c0148-7b31-468f-a949-6851c247636f-tmp\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.147059 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.147031 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28tc\" (UniqueName: \"kubernetes.io/projected/a80c0148-7b31-468f-a949-6851c247636f-kube-api-access-w28tc\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.445588 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.445549 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" event={"ID":"32145e5f-b3cc-40b9-b055-684bee700c51","Type":"ContainerStarted","Data":"74a279b3afd96f2f65f432fabeb60b92ede4caaf18ebeb5e5c77e05ef4cf0ee1"} Jul 10 14:40:44.617114 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.617074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.620302 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.620250 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a80c0148-7b31-468f-a949-6851c247636f-tls-certs\") pod \"kube-auth-proxy-74544d9668-hvgp4\" (UID: \"a80c0148-7b31-468f-a949-6851c247636f\") " pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:44.822148 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:44.822029 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" Jul 10 14:40:45.055213 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:45.055070 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74544d9668-hvgp4"] Jul 10 14:40:45.058458 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:40:45.058419 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80c0148_7b31_468f_a949_6851c247636f.slice/crio-9160ad2f7c2076e07fca750ad8785516dd796937a062fc01762371aca18e4b04 WatchSource:0}: Error finding container 9160ad2f7c2076e07fca750ad8785516dd796937a062fc01762371aca18e4b04: Status 404 returned error can't find the container with id 9160ad2f7c2076e07fca750ad8785516dd796937a062fc01762371aca18e4b04 Jul 10 14:40:45.450585 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:45.450513 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" event={"ID":"a80c0148-7b31-468f-a949-6851c247636f","Type":"ContainerStarted","Data":"9160ad2f7c2076e07fca750ad8785516dd796937a062fc01762371aca18e4b04"} Jul 10 14:40:46.456620 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:46.456588 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" event={"ID":"32145e5f-b3cc-40b9-b055-684bee700c51","Type":"ContainerStarted","Data":"6635d3a2c8e10c3b9251ee2e6dd65d8172e1f3d6531803d0762bc5ef30b366fe"} Jul 10 14:40:46.457056 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:46.456716 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:40:46.485605 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:46.485556 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" podStartSLOduration=1.048754659 podStartE2EDuration="3.485534653s" totalImagesPullingTime="2.436779994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:46.483143247 +0000 UTC m=+336.077498790" watchObservedRunningTime="2026-07-10 14:40:46.485534653 +0000 UTC m=+336.079890176" Jul 10 14:40:48.465165 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:48.465127 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" event={"ID":"a80c0148-7b31-468f-a949-6851c247636f","Type":"ContainerStarted","Data":"bc1462f2d3b2afe324940f876a05317f323922f6adc2a9969e4af7bc3537f41c"} Jul 10 14:40:48.487504 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:48.487455 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74544d9668-hvgp4" podStartSLOduration=2.171365692 podStartE2EDuration="5.487439153s" totalImagesPullingTime="3.316073461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:40:48.486392904 +0000 UTC m=+338.080748423" watchObservedRunningTime="2026-07-10 14:40:48.487439153 +0000 UTC m=+338.081794726" Jul 10 14:40:57.462965 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:40:57.462931 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-869f49798c-vvj7c" Jul 10 14:42:43.161813 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.161724 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm"] Jul 10 14:42:43.164591 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.164573 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.167700 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.167679 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-28lrd\"" Jul 10 14:42:43.167826 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.167808 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 10 14:42:43.168340 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.168325 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 10 14:42:43.168985 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.168969 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 14:42:43.169354 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.169339 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 14:42:43.182802 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.182773 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm"] Jul 10 14:42:43.240280 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.240249 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a45f9c62-b38b-4c5a-843a-af00b44de9a2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.240280 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.240283 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrjp6\" (UniqueName: \"kubernetes.io/projected/a45f9c62-b38b-4c5a-843a-af00b44de9a2-kube-api-access-wrjp6\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.240487 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.240317 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45f9c62-b38b-4c5a-843a-af00b44de9a2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.341071 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.341037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a45f9c62-b38b-4c5a-843a-af00b44de9a2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.341250 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.341092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrjp6\" (UniqueName: \"kubernetes.io/projected/a45f9c62-b38b-4c5a-843a-af00b44de9a2-kube-api-access-wrjp6\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.341250 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.341148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45f9c62-b38b-4c5a-843a-af00b44de9a2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.341705 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.341687 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a45f9c62-b38b-4c5a-843a-af00b44de9a2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.343772 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.343747 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a45f9c62-b38b-4c5a-843a-af00b44de9a2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.351048 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.351025 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrjp6\" (UniqueName: \"kubernetes.io/projected/a45f9c62-b38b-4c5a-843a-af00b44de9a2-kube-api-access-wrjp6\") pod \"kuadrant-console-plugin-66b5774985-dkcfm\" (UID: \"a45f9c62-b38b-4c5a-843a-af00b44de9a2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.473672 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.473602 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" Jul 10 14:42:43.603812 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.603781 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm"] Jul 10 14:42:43.606910 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:42:43.606878 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda45f9c62_b38b_4c5a_843a_af00b44de9a2.slice/crio-46daef4b8795226fe87b37ae12466079d97178da9bc033b8c59122ef5b58ee48 WatchSource:0}: Error finding container 46daef4b8795226fe87b37ae12466079d97178da9bc033b8c59122ef5b58ee48: Status 404 returned error can't find the container with id 46daef4b8795226fe87b37ae12466079d97178da9bc033b8c59122ef5b58ee48 Jul 10 14:42:43.878648 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:42:43.878611 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" event={"ID":"a45f9c62-b38b-4c5a-843a-af00b44de9a2","Type":"ContainerStarted","Data":"46daef4b8795226fe87b37ae12466079d97178da9bc033b8c59122ef5b58ee48"} Jul 10 14:43:05.985050 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:05.984955 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" event={"ID":"a45f9c62-b38b-4c5a-843a-af00b44de9a2","Type":"ContainerStarted","Data":"dc536b29d68e7575b0e13c1d347ddab5ce1472eaa85e60b079d6eec8a8325b83"} Jul 10 14:43:06.002739 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:06.002679 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dkcfm" podStartSLOduration=0.95250498 podStartE2EDuration="23.002635812s" totalImagesPullingTime="22.050130832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:06.002424827 +0000 UTC m=+475.596780343" watchObservedRunningTime="2026-07-10 14:43:06.002635812 +0000 UTC m=+475.596991335" Jul 10 14:43:42.887808 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:42.887777 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:42.965384 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:42.965343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:42.965543 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:42.965440 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:42.968388 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:42.968365 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-scbvb\"" Jul 10 14:43:43.027020 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.026976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm4sx\" (UniqueName: \"kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx\") pod \"authorino-58cd955675-j4vsb\" (UID: \"1265366e-5a03-4e7a-8675-08b3acac4c74\") " pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:43.092972 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.092937 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:43:43.096184 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.096167 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:43:43.105666 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.105642 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:43:43.127406 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.127380 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm4sx\" (UniqueName: \"kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx\") pod \"authorino-58cd955675-j4vsb\" (UID: \"1265366e-5a03-4e7a-8675-08b3acac4c74\") " pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:43.127553 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.127439 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz\") pod \"authorino-75d556bd77-b7rr2\" (UID: \"5f50611b-dba7-476c-bd12-65f51203754f\") " pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:43:43.139102 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.139043 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm4sx\" (UniqueName: \"kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx\") pod \"authorino-58cd955675-j4vsb\" (UID: \"1265366e-5a03-4e7a-8675-08b3acac4c74\") " pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:43.228809 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.228766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz\") pod \"authorino-75d556bd77-b7rr2\" (UID: \"5f50611b-dba7-476c-bd12-65f51203754f\") " pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:43:43.240107 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.240080 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz\") pod \"authorino-75d556bd77-b7rr2\" (UID: \"5f50611b-dba7-476c-bd12-65f51203754f\") " pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:43:43.273838 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.273805 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:43.404462 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.404387 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:43:43.405521 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.405498 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:43.408130 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:43:43.408098 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1265366e_5a03_4e7a_8675_08b3acac4c74.slice/crio-ef1c05cf10b26dd9d455a34b57c1316a83df5a71dca4185e8c63c19a084c2c50 WatchSource:0}: Error finding container ef1c05cf10b26dd9d455a34b57c1316a83df5a71dca4185e8c63c19a084c2c50: Status 404 returned error can't find the container with id ef1c05cf10b26dd9d455a34b57c1316a83df5a71dca4185e8c63c19a084c2c50 Jul 10 14:43:43.539021 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:43.539003 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:43:43.541645 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:43:43.541623 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f50611b_dba7_476c_bd12_65f51203754f.slice/crio-818d59b2d90a83e82a89c519e674587aa19e6f6c3d10a4777325eaa0cc3fa5a7 WatchSource:0}: Error finding container 818d59b2d90a83e82a89c519e674587aa19e6f6c3d10a4777325eaa0cc3fa5a7: Status 404 returned error can't find the container with id 818d59b2d90a83e82a89c519e674587aa19e6f6c3d10a4777325eaa0cc3fa5a7 Jul 10 14:43:44.147565 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:44.147399 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j4vsb" event={"ID":"1265366e-5a03-4e7a-8675-08b3acac4c74","Type":"ContainerStarted","Data":"ef1c05cf10b26dd9d455a34b57c1316a83df5a71dca4185e8c63c19a084c2c50"} Jul 10 14:43:44.150825 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:44.150762 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b7rr2" event={"ID":"5f50611b-dba7-476c-bd12-65f51203754f","Type":"ContainerStarted","Data":"818d59b2d90a83e82a89c519e674587aa19e6f6c3d10a4777325eaa0cc3fa5a7"} Jul 10 14:43:47.166344 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:47.166245 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j4vsb" event={"ID":"1265366e-5a03-4e7a-8675-08b3acac4c74","Type":"ContainerStarted","Data":"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29"} Jul 10 14:43:47.167618 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:47.167593 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b7rr2" event={"ID":"5f50611b-dba7-476c-bd12-65f51203754f","Type":"ContainerStarted","Data":"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1"} Jul 10 14:43:47.200607 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:47.200557 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-j4vsb" podStartSLOduration=1.7561478689999999 podStartE2EDuration="5.200544022s" totalImagesPullingTime="3.444396153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:47.200336709 +0000 UTC m=+516.794692228" watchObservedRunningTime="2026-07-10 14:43:47.200544022 +0000 UTC m=+516.794899541" Jul 10 14:43:47.236713 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:47.236664 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-b7rr2" podStartSLOduration=0.915202463 podStartE2EDuration="4.236646899s" totalImagesPullingTime="3.321444436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:43:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:43:47.235373427 +0000 UTC m=+516.829728951" watchObservedRunningTime="2026-07-10 14:43:47.236646899 +0000 UTC m=+516.831002473" Jul 10 14:43:47.282234 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:47.282182 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:49.175952 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:49.175907 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-j4vsb" podUID="1265366e-5a03-4e7a-8675-08b3acac4c74" containerName="authorino" containerID="cri-o://91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29" gracePeriod=30 Jul 10 14:43:49.427642 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:49.427577 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:49.482513 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:49.482470 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm4sx\" (UniqueName: \"kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx\") pod \"1265366e-5a03-4e7a-8675-08b3acac4c74\" (UID: \"1265366e-5a03-4e7a-8675-08b3acac4c74\") " Jul 10 14:43:49.484788 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:49.484762 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx" (OuterVolumeSpecName: "kube-api-access-qm4sx") pod "1265366e-5a03-4e7a-8675-08b3acac4c74" (UID: "1265366e-5a03-4e7a-8675-08b3acac4c74"). InnerVolumeSpecName "kube-api-access-qm4sx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:43:49.583643 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:49.583596 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm4sx\" (UniqueName: \"kubernetes.io/projected/1265366e-5a03-4e7a-8675-08b3acac4c74-kube-api-access-qm4sx\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:43:50.181013 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.180977 2572 generic.go:356] "Generic (PLEG): container finished" podID="1265366e-5a03-4e7a-8675-08b3acac4c74" containerID="91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29" exitCode=0 Jul 10 14:43:50.181013 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.181017 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-j4vsb" Jul 10 14:43:50.181548 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.181060 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j4vsb" event={"ID":"1265366e-5a03-4e7a-8675-08b3acac4c74","Type":"ContainerDied","Data":"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29"} Jul 10 14:43:50.181548 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.181095 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-j4vsb" event={"ID":"1265366e-5a03-4e7a-8675-08b3acac4c74","Type":"ContainerDied","Data":"ef1c05cf10b26dd9d455a34b57c1316a83df5a71dca4185e8c63c19a084c2c50"} Jul 10 14:43:50.181548 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.181110 2572 scope.go:117] "RemoveContainer" containerID="91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29" Jul 10 14:43:50.189856 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.189835 2572 scope.go:117] "RemoveContainer" containerID="91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29" Jul 10 14:43:50.190163 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:43:50.190142 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29\": container with ID starting with 91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29 not found: ID does not exist" containerID="91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29" Jul 10 14:43:50.190272 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.190176 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29"} err="failed to get container status \"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29\": rpc error: code = NotFound desc = could not find container \"91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29\": container with ID starting with 91bc18cfe898416476fd31dd89bc69710c7d9502a4010bce1778246224728e29 not found: ID does not exist" Jul 10 14:43:50.201566 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.201533 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:50.212893 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:50.212856 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-j4vsb"] Jul 10 14:43:51.075398 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:43:51.075363 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1265366e-5a03-4e7a-8675-08b3acac4c74" path="/var/lib/kubelet/pods/1265366e-5a03-4e7a-8675-08b3acac4c74/volumes" Jul 10 14:44:13.006771 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.006728 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:13.007286 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.007133 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1265366e-5a03-4e7a-8675-08b3acac4c74" containerName="authorino" Jul 10 14:44:13.007286 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.007151 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1265366e-5a03-4e7a-8675-08b3acac4c74" containerName="authorino" Jul 10 14:44:13.007286 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.007239 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1265366e-5a03-4e7a-8675-08b3acac4c74" containerName="authorino" Jul 10 14:44:13.009565 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.009544 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:13.018771 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.018745 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:13.065637 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.065603 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkm7m\" (UniqueName: \"kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m\") pod \"authorino-5d4b67c446-blk72\" (UID: \"4495de70-02b0-4f37-995c-30e86c808c35\") " pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:13.166112 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.166070 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkm7m\" (UniqueName: \"kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m\") pod \"authorino-5d4b67c446-blk72\" (UID: \"4495de70-02b0-4f37-995c-30e86c808c35\") " pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:13.175869 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.175839 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkm7m\" (UniqueName: \"kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m\") pod \"authorino-5d4b67c446-blk72\" (UID: \"4495de70-02b0-4f37-995c-30e86c808c35\") " pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:13.246021 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.245985 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:13.246225 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.246213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:13.280627 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.280543 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:13.282691 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.282670 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:13.291829 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.291801 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:13.367871 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.367841 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg5rv\" (UniqueName: \"kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv\") pod \"authorino-665d845894-rj9cz\" (UID: \"aa4dead5-a794-41df-b0e5-87ba1b7777e8\") " pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:13.380979 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.380955 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:13.383119 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:44:13.383091 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4495de70_02b0_4f37_995c_30e86c808c35.slice/crio-c08e22502e446ba86c0cdcddc776c5d1821ff489f8484b2db3443107050c4cea WatchSource:0}: Error finding container c08e22502e446ba86c0cdcddc776c5d1821ff489f8484b2db3443107050c4cea: Status 404 returned error can't find the container with id c08e22502e446ba86c0cdcddc776c5d1821ff489f8484b2db3443107050c4cea Jul 10 14:44:13.469139 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.469113 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg5rv\" (UniqueName: \"kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv\") pod \"authorino-665d845894-rj9cz\" (UID: \"aa4dead5-a794-41df-b0e5-87ba1b7777e8\") " pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:13.478598 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.478569 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg5rv\" (UniqueName: \"kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv\") pod \"authorino-665d845894-rj9cz\" (UID: \"aa4dead5-a794-41df-b0e5-87ba1b7777e8\") " pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:13.592394 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.592361 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:13.628598 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.624284 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:13.660542 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.660509 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:44:13.664384 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.664357 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.667588 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.667561 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 14:44:13.673474 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.673446 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:44:13.735211 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.735160 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:13.737769 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:44:13.737730 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa4dead5_a794_41df_b0e5_87ba1b7777e8.slice/crio-4ef8989048803b57fa1bf32baf83e8c7f3aedc977616ffe8fb67128fd802c2db WatchSource:0}: Error finding container 4ef8989048803b57fa1bf32baf83e8c7f3aedc977616ffe8fb67128fd802c2db: Status 404 returned error can't find the container with id 4ef8989048803b57fa1bf32baf83e8c7f3aedc977616ffe8fb67128fd802c2db Jul 10 14:44:13.770777 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.770745 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599tn\" (UniqueName: \"kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.770935 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.770896 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.871590 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.871494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.871590 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.871534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599tn\" (UniqueName: \"kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.874704 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.874672 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.885457 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.885430 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-599tn\" (UniqueName: \"kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn\") pod \"authorino-595d89d474-s5tzc\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:13.974987 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:13.974955 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:44:14.119911 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.119883 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:44:14.125376 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:44:14.125354 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde8ce056_5254_41be_86c0_6dbd7feacdc9.slice/crio-3599a167d0d3ee1e6aee52c344c278651d3ff1f6de9b3be0dbc64eaa64a52791 WatchSource:0}: Error finding container 3599a167d0d3ee1e6aee52c344c278651d3ff1f6de9b3be0dbc64eaa64a52791: Status 404 returned error can't find the container with id 3599a167d0d3ee1e6aee52c344c278651d3ff1f6de9b3be0dbc64eaa64a52791 Jul 10 14:44:14.276007 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.275966 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665d845894-rj9cz" event={"ID":"aa4dead5-a794-41df-b0e5-87ba1b7777e8","Type":"ContainerStarted","Data":"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062"} Jul 10 14:44:14.276007 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.276012 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665d845894-rj9cz" event={"ID":"aa4dead5-a794-41df-b0e5-87ba1b7777e8","Type":"ContainerStarted","Data":"4ef8989048803b57fa1bf32baf83e8c7f3aedc977616ffe8fb67128fd802c2db"} Jul 10 14:44:14.276275 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.276007 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-665d845894-rj9cz" podUID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" containerName="authorino" containerID="cri-o://a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062" gracePeriod=30 Jul 10 14:44:14.277400 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.277378 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d89d474-s5tzc" event={"ID":"de8ce056-5254-41be-86c0-6dbd7feacdc9","Type":"ContainerStarted","Data":"3599a167d0d3ee1e6aee52c344c278651d3ff1f6de9b3be0dbc64eaa64a52791"} Jul 10 14:44:14.278932 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.278907 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-blk72" event={"ID":"4495de70-02b0-4f37-995c-30e86c808c35","Type":"ContainerStarted","Data":"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee"} Jul 10 14:44:14.279047 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.278943 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-blk72" event={"ID":"4495de70-02b0-4f37-995c-30e86c808c35","Type":"ContainerStarted","Data":"c08e22502e446ba86c0cdcddc776c5d1821ff489f8484b2db3443107050c4cea"} Jul 10 14:44:14.279047 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.278956 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-blk72" podUID="4495de70-02b0-4f37-995c-30e86c808c35" containerName="authorino" containerID="cri-o://36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee" gracePeriod=30 Jul 10 14:44:14.294617 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.294479 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-665d845894-rj9cz" podStartSLOduration=0.956687141 podStartE2EDuration="1.294461893s" totalImagesPullingTime="337.774752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:14.293661235 +0000 UTC m=+543.888016754" watchObservedRunningTime="2026-07-10 14:44:14.294461893 +0000 UTC m=+543.888817423" Jul 10 14:44:14.312358 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.312312 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-blk72" podStartSLOduration=1.957768271 podStartE2EDuration="2.312294628s" totalImagesPullingTime="354.526357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:14.311179111 +0000 UTC m=+543.905534643" watchObservedRunningTime="2026-07-10 14:44:14.312294628 +0000 UTC m=+543.906650147" Jul 10 14:44:14.535688 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.535663 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:14.550373 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.550351 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:14.577461 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.577404 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkm7m\" (UniqueName: \"kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m\") pod \"4495de70-02b0-4f37-995c-30e86c808c35\" (UID: \"4495de70-02b0-4f37-995c-30e86c808c35\") " Jul 10 14:44:14.579618 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.579591 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m" (OuterVolumeSpecName: "kube-api-access-hkm7m") pod "4495de70-02b0-4f37-995c-30e86c808c35" (UID: "4495de70-02b0-4f37-995c-30e86c808c35"). InnerVolumeSpecName "kube-api-access-hkm7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:44:14.678937 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.678852 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg5rv\" (UniqueName: \"kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv\") pod \"aa4dead5-a794-41df-b0e5-87ba1b7777e8\" (UID: \"aa4dead5-a794-41df-b0e5-87ba1b7777e8\") " Jul 10 14:44:14.679078 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.679047 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkm7m\" (UniqueName: \"kubernetes.io/projected/4495de70-02b0-4f37-995c-30e86c808c35-kube-api-access-hkm7m\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:44:14.681027 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.680995 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv" (OuterVolumeSpecName: "kube-api-access-hg5rv") pod "aa4dead5-a794-41df-b0e5-87ba1b7777e8" (UID: "aa4dead5-a794-41df-b0e5-87ba1b7777e8"). InnerVolumeSpecName "kube-api-access-hg5rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:44:14.780132 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:14.780099 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg5rv\" (UniqueName: \"kubernetes.io/projected/aa4dead5-a794-41df-b0e5-87ba1b7777e8-kube-api-access-hg5rv\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:44:15.284780 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.284748 2572 generic.go:356] "Generic (PLEG): container finished" podID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" containerID="a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062" exitCode=0 Jul 10 14:44:15.285232 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.284789 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665d845894-rj9cz" event={"ID":"aa4dead5-a794-41df-b0e5-87ba1b7777e8","Type":"ContainerDied","Data":"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062"} Jul 10 14:44:15.285232 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.284828 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665d845894-rj9cz" event={"ID":"aa4dead5-a794-41df-b0e5-87ba1b7777e8","Type":"ContainerDied","Data":"4ef8989048803b57fa1bf32baf83e8c7f3aedc977616ffe8fb67128fd802c2db"} Jul 10 14:44:15.285232 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.284797 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665d845894-rj9cz" Jul 10 14:44:15.285232 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.284850 2572 scope.go:117] "RemoveContainer" containerID="a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062" Jul 10 14:44:15.286579 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.286552 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d89d474-s5tzc" event={"ID":"de8ce056-5254-41be-86c0-6dbd7feacdc9","Type":"ContainerStarted","Data":"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820"} Jul 10 14:44:15.288172 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.288147 2572 generic.go:356] "Generic (PLEG): container finished" podID="4495de70-02b0-4f37-995c-30e86c808c35" containerID="36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee" exitCode=0 Jul 10 14:44:15.288295 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.288183 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-blk72" Jul 10 14:44:15.288295 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.288225 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-blk72" event={"ID":"4495de70-02b0-4f37-995c-30e86c808c35","Type":"ContainerDied","Data":"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee"} Jul 10 14:44:15.288295 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.288256 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-blk72" event={"ID":"4495de70-02b0-4f37-995c-30e86c808c35","Type":"ContainerDied","Data":"c08e22502e446ba86c0cdcddc776c5d1821ff489f8484b2db3443107050c4cea"} Jul 10 14:44:15.293625 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.293608 2572 scope.go:117] "RemoveContainer" containerID="a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062" Jul 10 14:44:15.293891 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:44:15.293871 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062\": container with ID starting with a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062 not found: ID does not exist" containerID="a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062" Jul 10 14:44:15.293979 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.293901 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062"} err="failed to get container status \"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062\": rpc error: code = NotFound desc = could not find container \"a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062\": container with ID starting with a169ee34c37e5cdd65a1960b92d10d8140f4969d505490484b353d164a5f8062 not found: ID does not exist" Jul 10 14:44:15.293979 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.293922 2572 scope.go:117] "RemoveContainer" containerID="36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee" Jul 10 14:44:15.301239 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.301222 2572 scope.go:117] "RemoveContainer" containerID="36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee" Jul 10 14:44:15.301493 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:44:15.301476 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee\": container with ID starting with 36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee not found: ID does not exist" containerID="36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee" Jul 10 14:44:15.301559 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.301499 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee"} err="failed to get container status \"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee\": rpc error: code = NotFound desc = could not find container \"36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee\": container with ID starting with 36b99b4c0cd6140597e445463ca3eebdec1f3ed5587fe97f5deedb66bb65beee not found: ID does not exist" Jul 10 14:44:15.307611 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.307570 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-595d89d474-s5tzc" podStartSLOduration=1.924873995 podStartE2EDuration="2.307557663s" totalImagesPullingTime="382.683668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:15.307365723 +0000 UTC m=+544.901721236" watchObservedRunningTime="2026-07-10 14:44:15.307557663 +0000 UTC m=+544.901913182" Jul 10 14:44:15.327473 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.327437 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:15.333783 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.333738 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-665d845894-rj9cz"] Jul 10 14:44:15.338374 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.338309 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:44:15.338536 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.338513 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-b7rr2" podUID="5f50611b-dba7-476c-bd12-65f51203754f" containerName="authorino" containerID="cri-o://04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1" gracePeriod=30 Jul 10 14:44:15.351140 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.351112 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:15.357919 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.357898 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-blk72"] Jul 10 14:44:15.578800 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.578776 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:44:15.686834 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.686748 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz\") pod \"5f50611b-dba7-476c-bd12-65f51203754f\" (UID: \"5f50611b-dba7-476c-bd12-65f51203754f\") " Jul 10 14:44:15.688870 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.688847 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz" (OuterVolumeSpecName: "kube-api-access-xjnjz") pod "5f50611b-dba7-476c-bd12-65f51203754f" (UID: "5f50611b-dba7-476c-bd12-65f51203754f"). InnerVolumeSpecName "kube-api-access-xjnjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:44:15.788242 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:15.788171 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/5f50611b-dba7-476c-bd12-65f51203754f-kube-api-access-xjnjz\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:44:16.295376 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.295345 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f50611b-dba7-476c-bd12-65f51203754f" containerID="04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1" exitCode=0 Jul 10 14:44:16.295376 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.295385 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b7rr2" Jul 10 14:44:16.295927 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.295420 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b7rr2" event={"ID":"5f50611b-dba7-476c-bd12-65f51203754f","Type":"ContainerDied","Data":"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1"} Jul 10 14:44:16.295927 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.295465 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b7rr2" event={"ID":"5f50611b-dba7-476c-bd12-65f51203754f","Type":"ContainerDied","Data":"818d59b2d90a83e82a89c519e674587aa19e6f6c3d10a4777325eaa0cc3fa5a7"} Jul 10 14:44:16.295927 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.295487 2572 scope.go:117] "RemoveContainer" containerID="04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1" Jul 10 14:44:16.303746 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.303730 2572 scope.go:117] "RemoveContainer" containerID="04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1" Jul 10 14:44:16.303963 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:44:16.303948 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1\": container with ID starting with 04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1 not found: ID does not exist" containerID="04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1" Jul 10 14:44:16.304013 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.303969 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1"} err="failed to get container status \"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1\": rpc error: code = NotFound desc = could not find container \"04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1\": container with ID starting with 04d33b944f7dedea90d4ddab82adcd00fe7524d97e591822c2db0de84c6d1cd1 not found: ID does not exist" Jul 10 14:44:16.316809 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.316780 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:44:16.325385 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:16.322544 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b7rr2"] Jul 10 14:44:17.075796 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:17.075757 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4495de70-02b0-4f37-995c-30e86c808c35" path="/var/lib/kubelet/pods/4495de70-02b0-4f37-995c-30e86c808c35/volumes" Jul 10 14:44:17.076075 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:17.076063 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f50611b-dba7-476c-bd12-65f51203754f" path="/var/lib/kubelet/pods/5f50611b-dba7-476c-bd12-65f51203754f/volumes" Jul 10 14:44:17.076365 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:17.076353 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" path="/var/lib/kubelet/pods/aa4dead5-a794-41df-b0e5-87ba1b7777e8/volumes" Jul 10 14:44:56.944511 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944472 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5rrhf"] Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944792 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4495de70-02b0-4f37-995c-30e86c808c35" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944803 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4495de70-02b0-4f37-995c-30e86c808c35" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944815 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f50611b-dba7-476c-bd12-65f51203754f" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944821 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f50611b-dba7-476c-bd12-65f51203754f" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944838 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944845 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944921 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f50611b-dba7-476c-bd12-65f51203754f" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944929 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4495de70-02b0-4f37-995c-30e86c808c35" containerName="authorino" Jul 10 14:44:56.945012 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.944935 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4dead5-a794-41df-b0e5-87ba1b7777e8" containerName="authorino" Jul 10 14:44:56.948206 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.948175 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:56.951379 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.951354 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-z5wdk\"" Jul 10 14:44:56.951535 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.951512 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 14:44:56.957023 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:56.956996 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5rrhf"] Jul 10 14:44:57.022382 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.022348 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvvf\" (UniqueName: \"kubernetes.io/projected/17654171-fd04-4d6d-851e-23339f4cd211-kube-api-access-4cvvf\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.022614 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.022400 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17654171-fd04-4d6d-851e-23339f4cd211-plugins-config-volume\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.122846 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.122803 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvvf\" (UniqueName: \"kubernetes.io/projected/17654171-fd04-4d6d-851e-23339f4cd211-kube-api-access-4cvvf\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.123048 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.122855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17654171-fd04-4d6d-851e-23339f4cd211-plugins-config-volume\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.123613 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.123588 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17654171-fd04-4d6d-851e-23339f4cd211-plugins-config-volume\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.132061 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.132041 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvvf\" (UniqueName: \"kubernetes.io/projected/17654171-fd04-4d6d-851e-23339f4cd211-kube-api-access-4cvvf\") pod \"payload-processing-58f86c7798-5rrhf\" (UID: \"17654171-fd04-4d6d-851e-23339f4cd211\") " pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.259903 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.259800 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:57.393259 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.393228 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5rrhf"] Jul 10 14:44:57.396633 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:44:57.396583 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17654171_fd04_4d6d_851e_23339f4cd211.slice/crio-4563b4cb03e628c2327d853e77fa69ee78296bc76e7deebb02a5cda5f33830b2 WatchSource:0}: Error finding container 4563b4cb03e628c2327d853e77fa69ee78296bc76e7deebb02a5cda5f33830b2: Status 404 returned error can't find the container with id 4563b4cb03e628c2327d853e77fa69ee78296bc76e7deebb02a5cda5f33830b2 Jul 10 14:44:57.459293 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:57.459257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" event={"ID":"17654171-fd04-4d6d-851e-23339f4cd211","Type":"ContainerStarted","Data":"4563b4cb03e628c2327d853e77fa69ee78296bc76e7deebb02a5cda5f33830b2"} Jul 10 14:44:59.471173 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:59.471137 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" event={"ID":"17654171-fd04-4d6d-851e-23339f4cd211","Type":"ContainerStarted","Data":"b7fee596911e215ab973160af0f06b6dc849fdcb0a1afdfc7b96dc57a9607641"} Jul 10 14:44:59.471579 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:59.471268 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:44:59.491786 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:44:59.491739 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" podStartSLOduration=2.151736672 podStartE2EDuration="3.491721013s" totalImagesPullingTime="1.339984341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:44:59.490536038 +0000 UTC m=+589.084891559" watchObservedRunningTime="2026-07-10 14:44:59.491721013 +0000 UTC m=+589.086076532" Jul 10 14:45:10.476849 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:10.476816 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5rrhf" Jul 10 14:45:10.828873 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:10.828843 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:45:10.829904 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:10.829878 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:45:48.420081 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.420003 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b"] Jul 10 14:45:48.424385 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.424367 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.427756 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.427732 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 14:45:48.428807 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.428789 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 14:45:48.428869 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.428818 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wbj7h\"" Jul 10 14:45:48.428922 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.428904 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 14:45:48.436902 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.436878 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b"] Jul 10 14:45:48.550412 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.550581 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmcrw\" (UniqueName: \"kubernetes.io/projected/81bbce8b-1d31-40b7-899a-b6cb3db446ba-kube-api-access-qmcrw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.550581 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550507 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.550581 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.550696 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550589 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.550696 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.550603 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651243 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmcrw\" (UniqueName: \"kubernetes.io/projected/81bbce8b-1d31-40b7-899a-b6cb3db446ba-kube-api-access-qmcrw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651243 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651252 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651499 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651307 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651499 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651341 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651499 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651363 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651499 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651401 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651816 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651786 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651794 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.651896 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.651829 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.653721 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.653693 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81bbce8b-1d31-40b7-899a-b6cb3db446ba-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.653893 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.653876 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81bbce8b-1d31-40b7-899a-b6cb3db446ba-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.661303 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.661275 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmcrw\" (UniqueName: \"kubernetes.io/projected/81bbce8b-1d31-40b7-899a-b6cb3db446ba-kube-api-access-qmcrw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-m5c6b\" (UID: \"81bbce8b-1d31-40b7-899a-b6cb3db446ba\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.734889 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.734793 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:48.874719 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.874688 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b"] Jul 10 14:45:48.877941 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:45:48.877916 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81bbce8b_1d31_40b7_899a_b6cb3db446ba.slice/crio-c72ff6459b062be4a3ba83ba30352c3d17cae4ce272aee958b19e266055e7723 WatchSource:0}: Error finding container c72ff6459b062be4a3ba83ba30352c3d17cae4ce272aee958b19e266055e7723: Status 404 returned error can't find the container with id c72ff6459b062be4a3ba83ba30352c3d17cae4ce272aee958b19e266055e7723 Jul 10 14:45:48.879225 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:48.879184 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 14:45:49.669614 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:49.669576 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" event={"ID":"81bbce8b-1d31-40b7-899a-b6cb3db446ba","Type":"ContainerStarted","Data":"c72ff6459b062be4a3ba83ba30352c3d17cae4ce272aee958b19e266055e7723"} Jul 10 14:45:50.675214 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:50.675163 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" event={"ID":"81bbce8b-1d31-40b7-899a-b6cb3db446ba","Type":"ContainerStarted","Data":"59b6b4bf5fdfc3cae0f74f1380efa3209a2e464ed8c27356968beebb5c049686"} Jul 10 14:45:50.695554 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:50.695508 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" podStartSLOduration=1.572686514 podStartE2EDuration="2.695492994s" totalImagesPullingTime="1.12280648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:45:50.695093719 +0000 UTC m=+640.289449432" watchObservedRunningTime="2026-07-10 14:45:50.695492994 +0000 UTC m=+640.289848512" Jul 10 14:45:58.735113 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:58.735070 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:58.735630 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:58.735144 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:58.747723 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:58.747696 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:45:59.723187 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:45:59.723163 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-m5c6b" Jul 10 14:46:24.398652 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.398617 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz"] Jul 10 14:46:24.404364 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.404346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.408029 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.408010 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 14:46:24.417224 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.417181 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz"] Jul 10 14:46:24.537061 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537027 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/816f639d-54b2-400d-b92e-efafc9dc27f8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.537061 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537071 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.537290 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.537290 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537145 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.537290 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537171 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbhq\" (UniqueName: \"kubernetes.io/projected/816f639d-54b2-400d-b92e-efafc9dc27f8-kube-api-access-lqbhq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.537290 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.537245 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638085 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638048 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638085 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbhq\" (UniqueName: \"kubernetes.io/projected/816f639d-54b2-400d-b92e-efafc9dc27f8-kube-api-access-lqbhq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638229 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638319 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638304 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/816f639d-54b2-400d-b92e-efafc9dc27f8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638421 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638347 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638571 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638547 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638619 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638589 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.638677 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.638654 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.640654 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.640624 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/816f639d-54b2-400d-b92e-efafc9dc27f8-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.640853 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.640834 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/816f639d-54b2-400d-b92e-efafc9dc27f8-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.647813 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.647790 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbhq\" (UniqueName: \"kubernetes.io/projected/816f639d-54b2-400d-b92e-efafc9dc27f8-kube-api-access-lqbhq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vtstz\" (UID: \"816f639d-54b2-400d-b92e-efafc9dc27f8\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.714069 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.713990 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:24.848008 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:24.847976 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz"] Jul 10 14:46:24.851343 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:46:24.851315 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816f639d_54b2_400d_b92e_efafc9dc27f8.slice/crio-df498a7957d9765d457afdc9acab85e38c293b5bcb2a196b10785f39d0a9159e WatchSource:0}: Error finding container df498a7957d9765d457afdc9acab85e38c293b5bcb2a196b10785f39d0a9159e: Status 404 returned error can't find the container with id df498a7957d9765d457afdc9acab85e38c293b5bcb2a196b10785f39d0a9159e Jul 10 14:46:25.828151 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:25.828103 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" event={"ID":"816f639d-54b2-400d-b92e-efafc9dc27f8","Type":"ContainerStarted","Data":"d527f5d45f258a160e44faccc4aadb345b0b44e9a86ba89c5b0933116fb49652"} Jul 10 14:46:25.828559 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:25.828184 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" event={"ID":"816f639d-54b2-400d-b92e-efafc9dc27f8","Type":"ContainerStarted","Data":"df498a7957d9765d457afdc9acab85e38c293b5bcb2a196b10785f39d0a9159e"} Jul 10 14:46:25.863750 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:25.863682 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" podStartSLOduration=1.6450611469999998 podStartE2EDuration="1.863660037s" totalImagesPullingTime="218.59889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:25.860676797 +0000 UTC m=+675.455032331" watchObservedRunningTime="2026-07-10 14:46:25.863660037 +0000 UTC m=+675.458015559" Jul 10 14:46:31.190039 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.190005 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd"] Jul 10 14:46:31.193467 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.193444 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.196881 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.196861 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 10 14:46:31.207709 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.207683 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd"] Jul 10 14:46:31.294679 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294644 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.294854 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294695 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.294854 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294723 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqrp4\" (UniqueName: \"kubernetes.io/projected/ff06230d-7954-4ed4-ac3f-df9a09a734dd-kube-api-access-sqrp4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.294854 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.294854 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294767 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.294854 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.294803 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395574 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395535 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395574 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqrp4\" (UniqueName: \"kubernetes.io/projected/ff06230d-7954-4ed4-ac3f-df9a09a734dd-kube-api-access-sqrp4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395852 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395639 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395852 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395691 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395852 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395713 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.395852 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.395811 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.396038 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.396020 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.396142 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.396122 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.396327 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.396306 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.398204 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.398159 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff06230d-7954-4ed4-ac3f-df9a09a734dd-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.398496 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.398481 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff06230d-7954-4ed4-ac3f-df9a09a734dd-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.404538 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.404517 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqrp4\" (UniqueName: \"kubernetes.io/projected/ff06230d-7954-4ed4-ac3f-df9a09a734dd-kube-api-access-sqrp4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd\" (UID: \"ff06230d-7954-4ed4-ac3f-df9a09a734dd\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.502279 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.502177 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:31.635473 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.635446 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd"] Jul 10 14:46:31.637826 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:46:31.637795 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff06230d_7954_4ed4_ac3f_df9a09a734dd.slice/crio-3cc90e8cd3f686df28c197db28dfe1b0d0e0985c151963db95b176f9334f1e5e WatchSource:0}: Error finding container 3cc90e8cd3f686df28c197db28dfe1b0d0e0985c151963db95b176f9334f1e5e: Status 404 returned error can't find the container with id 3cc90e8cd3f686df28c197db28dfe1b0d0e0985c151963db95b176f9334f1e5e Jul 10 14:46:31.855561 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:31.855523 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" event={"ID":"ff06230d-7954-4ed4-ac3f-df9a09a734dd","Type":"ContainerStarted","Data":"3cc90e8cd3f686df28c197db28dfe1b0d0e0985c151963db95b176f9334f1e5e"} Jul 10 14:46:32.860384 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:32.860343 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" event={"ID":"ff06230d-7954-4ed4-ac3f-df9a09a734dd","Type":"ContainerStarted","Data":"dcacc1180e7875bcee9cd4f1c7974767945eab1636910b1c763c9ba250f1e951"} Jul 10 14:46:32.881926 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:32.881869 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" podStartSLOduration=1.63486157 podStartE2EDuration="1.88184927s" totalImagesPullingTime="246.9877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:46:32.880465698 +0000 UTC m=+682.474821217" watchObservedRunningTime="2026-07-10 14:46:32.88184927 +0000 UTC m=+682.476204788" Jul 10 14:46:34.714714 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:34.714661 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:34.715139 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:34.714752 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:34.732885 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:34.732856 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:34.881872 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:34.881843 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vtstz" Jul 10 14:46:41.502974 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:41.502921 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:41.503406 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:41.502992 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:41.515340 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:41.515314 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:46:41.911932 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:46:41.911904 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd" Jul 10 14:47:30.388391 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.388351 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59746674dc-wv6dn"] Jul 10 14:47:30.391167 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.391149 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.402364 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.402336 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59746674dc-wv6dn"] Jul 10 14:47:30.464672 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.464640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3e93a7a6-2730-4992-88b7-d5663928c8c5-tls-cert\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.464836 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.464687 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpm9h\" (UniqueName: \"kubernetes.io/projected/3e93a7a6-2730-4992-88b7-d5663928c8c5-kube-api-access-zpm9h\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.565757 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.565719 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3e93a7a6-2730-4992-88b7-d5663928c8c5-tls-cert\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.565946 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.565781 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpm9h\" (UniqueName: \"kubernetes.io/projected/3e93a7a6-2730-4992-88b7-d5663928c8c5-kube-api-access-zpm9h\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.568593 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.568565 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3e93a7a6-2730-4992-88b7-d5663928c8c5-tls-cert\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.575016 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.574982 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpm9h\" (UniqueName: \"kubernetes.io/projected/3e93a7a6-2730-4992-88b7-d5663928c8c5-kube-api-access-zpm9h\") pod \"authorino-59746674dc-wv6dn\" (UID: \"3e93a7a6-2730-4992-88b7-d5663928c8c5\") " pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.700027 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.699940 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59746674dc-wv6dn" Jul 10 14:47:30.846422 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:30.846399 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59746674dc-wv6dn"] Jul 10 14:47:30.849130 ip-10-0-136-92 kubenswrapper[2572]: W0710 14:47:30.849097 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e93a7a6_2730_4992_88b7_d5663928c8c5.slice/crio-1dcc518d99b7f8672e8caa1378013be3f88e8ef39c8622909f1b25491aaebb6a WatchSource:0}: Error finding container 1dcc518d99b7f8672e8caa1378013be3f88e8ef39c8622909f1b25491aaebb6a: Status 404 returned error can't find the container with id 1dcc518d99b7f8672e8caa1378013be3f88e8ef39c8622909f1b25491aaebb6a Jul 10 14:47:31.107615 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:31.107582 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59746674dc-wv6dn" event={"ID":"3e93a7a6-2730-4992-88b7-d5663928c8c5","Type":"ContainerStarted","Data":"1dcc518d99b7f8672e8caa1378013be3f88e8ef39c8622909f1b25491aaebb6a"} Jul 10 14:47:32.115590 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.115532 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59746674dc-wv6dn" event={"ID":"3e93a7a6-2730-4992-88b7-d5663928c8c5","Type":"ContainerStarted","Data":"ebb5c82a1daf0e50357e436fbc7c7223be4e8841cfd0c9838f10567964a720f0"} Jul 10 14:47:32.146090 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.146031 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59746674dc-wv6dn" podStartSLOduration=1.682674172 podStartE2EDuration="2.146011887s" totalImagesPullingTime="463.337715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 14:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 14:47:32.144055367 +0000 UTC m=+741.738410887" watchObservedRunningTime="2026-07-10 14:47:32.146011887 +0000 UTC m=+741.740367449" Jul 10 14:47:32.192814 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.192778 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:47:32.193006 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.192987 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-595d89d474-s5tzc" podUID="de8ce056-5254-41be-86c0-6dbd7feacdc9" containerName="authorino" containerID="cri-o://851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820" gracePeriod=30 Jul 10 14:47:32.450965 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.450928 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:47:32.486510 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.486475 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert\") pod \"de8ce056-5254-41be-86c0-6dbd7feacdc9\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " Jul 10 14:47:32.487948 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.487921 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-599tn\" (UniqueName: \"kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn\") pod \"de8ce056-5254-41be-86c0-6dbd7feacdc9\" (UID: \"de8ce056-5254-41be-86c0-6dbd7feacdc9\") " Jul 10 14:47:32.490280 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.490235 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn" (OuterVolumeSpecName: "kube-api-access-599tn") pod "de8ce056-5254-41be-86c0-6dbd7feacdc9" (UID: "de8ce056-5254-41be-86c0-6dbd7feacdc9"). InnerVolumeSpecName "kube-api-access-599tn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 14:47:32.499567 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.499533 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "de8ce056-5254-41be-86c0-6dbd7feacdc9" (UID: "de8ce056-5254-41be-86c0-6dbd7feacdc9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 14:47:32.589283 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.589236 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-599tn\" (UniqueName: \"kubernetes.io/projected/de8ce056-5254-41be-86c0-6dbd7feacdc9-kube-api-access-599tn\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:47:32.589283 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:32.589271 2572 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de8ce056-5254-41be-86c0-6dbd7feacdc9-tls-cert\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 14:47:33.122392 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.122351 2572 generic.go:356] "Generic (PLEG): container finished" podID="de8ce056-5254-41be-86c0-6dbd7feacdc9" containerID="851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820" exitCode=0 Jul 10 14:47:33.122810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.122445 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-595d89d474-s5tzc" Jul 10 14:47:33.122810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.122437 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d89d474-s5tzc" event={"ID":"de8ce056-5254-41be-86c0-6dbd7feacdc9","Type":"ContainerDied","Data":"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820"} Jul 10 14:47:33.122810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.122553 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-595d89d474-s5tzc" event={"ID":"de8ce056-5254-41be-86c0-6dbd7feacdc9","Type":"ContainerDied","Data":"3599a167d0d3ee1e6aee52c344c278651d3ff1f6de9b3be0dbc64eaa64a52791"} Jul 10 14:47:33.122810 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.122569 2572 scope.go:117] "RemoveContainer" containerID="851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820" Jul 10 14:47:33.131935 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.131914 2572 scope.go:117] "RemoveContainer" containerID="851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820" Jul 10 14:47:33.132244 ip-10-0-136-92 kubenswrapper[2572]: E0710 14:47:33.132218 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820\": container with ID starting with 851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820 not found: ID does not exist" containerID="851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820" Jul 10 14:47:33.132412 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.132247 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820"} err="failed to get container status \"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820\": rpc error: code = NotFound desc = could not find container \"851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820\": container with ID starting with 851d12566e1aa7198e00a6117f3d57dcf0142b07f44d37c6e15ae539e8ee0820 not found: ID does not exist" Jul 10 14:47:33.144065 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.144031 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:47:33.146928 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:33.146901 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-595d89d474-s5tzc"] Jul 10 14:47:35.076648 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:47:35.076615 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de8ce056-5254-41be-86c0-6dbd7feacdc9" path="/var/lib/kubelet/pods/de8ce056-5254-41be-86c0-6dbd7feacdc9/volumes" Jul 10 14:50:10.852952 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:50:10.852881 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:50:10.852952 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:50:10.852881 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:55:10.876103 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:55:10.876075 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 14:55:10.877406 ip-10-0-136-92 kubenswrapper[2572]: I0710 14:55:10.877386 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:00:10.900539 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:00:10.900508 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:00:10.902521 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:00:10.902498 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:05:10.925337 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:05:10.925307 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:05:10.927157 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:05:10.927134 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:07:39.502214 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.502164 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:07:39.502621 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.502524 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8ce056-5254-41be-86c0-6dbd7feacdc9" containerName="authorino" Jul 10 15:07:39.502621 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.502539 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8ce056-5254-41be-86c0-6dbd7feacdc9" containerName="authorino" Jul 10 15:07:39.502621 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.502594 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8ce056-5254-41be-86c0-6dbd7feacdc9" containerName="authorino" Jul 10 15:07:39.505459 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.505440 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.511896 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.511867 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff844d\"/\"openshift-service-ca.crt\"" Jul 10 15:07:39.512065 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.511920 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff844d\"/\"kube-root-ca.crt\"" Jul 10 15:07:39.512065 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.512036 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff844d\"/\"test-model-ff844d-kserve-self-signed-certs\"" Jul 10 15:07:39.512065 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.512046 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff844d\"/\"default-dockercfg-fj96v\"" Jul 10 15:07:39.523519 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.523476 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:07:39.646684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646636 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.646684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646688 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.646941 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646748 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.646941 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtgzx\" (UniqueName: \"kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.646941 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646804 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.646941 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.646868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748035 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.747995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748035 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748037 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748315 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748315 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748091 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtgzx\" (UniqueName: \"kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748315 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748315 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748142 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748580 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748549 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748701 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748594 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.748701 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.748608 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.750614 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.750582 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.750751 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.750734 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.758837 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.758775 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtgzx\" (UniqueName: \"kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx\") pod \"test-model-ff844d-kserve-65b5697489-9rk49\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.820623 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.820586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:39.974868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:39.974842 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:07:39.977500 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:07:39.977474 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7660b1ec_2d05_4d72_b36b_649164d81369.slice/crio-1b42b23bb14eb0ef17747c46e51dd0b903d87db523727427f6ede0251ee5f417 WatchSource:0}: Error finding container 1b42b23bb14eb0ef17747c46e51dd0b903d87db523727427f6ede0251ee5f417: Status 404 returned error can't find the container with id 1b42b23bb14eb0ef17747c46e51dd0b903d87db523727427f6ede0251ee5f417 Jul 10 15:07:40.210704 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:40.210666 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" event={"ID":"7660b1ec-2d05-4d72-b36b-649164d81369","Type":"ContainerStarted","Data":"6e56340303aedddc336807ff2543dc47c81f73d7606d3082074ddd3a9688dd53"} Jul 10 15:07:40.210704 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:40.210707 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" event={"ID":"7660b1ec-2d05-4d72-b36b-649164d81369","Type":"ContainerStarted","Data":"1b42b23bb14eb0ef17747c46e51dd0b903d87db523727427f6ede0251ee5f417"} Jul 10 15:07:40.234463 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:40.234411 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" podStartSLOduration=1.234394476 podStartE2EDuration="1.234394476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:07:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:07:40.233571467 +0000 UTC m=+1949.827926999" watchObservedRunningTime="2026-07-10 15:07:40.234394476 +0000 UTC m=+1949.828749994" Jul 10 15:07:49.821344 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:49.821308 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:49.821754 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:49.821360 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:49.833654 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:49.833628 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:07:50.271593 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:07:50.271557 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:08:02.218411 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:02.218325 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:08:17.239523 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.239470 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" podUID="7660b1ec-2d05-4d72-b36b-649164d81369" containerName="main" containerID="cri-o://6e56340303aedddc336807ff2543dc47c81f73d7606d3082074ddd3a9688dd53" gracePeriod=45 Jul 10 15:08:17.370875 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.370843 2572 generic.go:356] "Generic (PLEG): container finished" podID="7660b1ec-2d05-4d72-b36b-649164d81369" containerID="6e56340303aedddc336807ff2543dc47c81f73d7606d3082074ddd3a9688dd53" exitCode=0 Jul 10 15:08:17.371023 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.370926 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" event={"ID":"7660b1ec-2d05-4d72-b36b-649164d81369","Type":"ContainerDied","Data":"6e56340303aedddc336807ff2543dc47c81f73d7606d3082074ddd3a9688dd53"} Jul 10 15:08:17.492924 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.492860 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:08:17.563998 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.563968 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564171 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564026 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564171 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564055 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564171 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564098 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtgzx\" (UniqueName: \"kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564171 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564121 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564171 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564155 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm\") pod \"7660b1ec-2d05-4d72-b36b-649164d81369\" (UID: \"7660b1ec-2d05-4d72-b36b-649164d81369\") " Jul 10 15:08:17.564546 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564323 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home" (OuterVolumeSpecName: "home") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:08:17.564546 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564331 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:08:17.564546 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564451 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-tmp-dir\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:17.564546 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564444 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache" (OuterVolumeSpecName: "model-cache") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:08:17.564546 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.564464 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-home\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:17.566573 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.566547 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm" (OuterVolumeSpecName: "dshm") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:08:17.566651 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.566599 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx" (OuterVolumeSpecName: "kube-api-access-jtgzx") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "kube-api-access-jtgzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:08:17.566689 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.566651 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7660b1ec-2d05-4d72-b36b-649164d81369" (UID: "7660b1ec-2d05-4d72-b36b-649164d81369"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:08:17.664985 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.664948 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtgzx\" (UniqueName: \"kubernetes.io/projected/7660b1ec-2d05-4d72-b36b-649164d81369-kube-api-access-jtgzx\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:17.664985 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.664979 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-model-cache\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:17.664985 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.664990 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7660b1ec-2d05-4d72-b36b-649164d81369-dshm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:17.665220 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:17.664998 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7660b1ec-2d05-4d72-b36b-649164d81369-tls-certs\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:08:18.376390 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:18.376360 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" Jul 10 15:08:18.376832 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:18.376387 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49" event={"ID":"7660b1ec-2d05-4d72-b36b-649164d81369","Type":"ContainerDied","Data":"1b42b23bb14eb0ef17747c46e51dd0b903d87db523727427f6ede0251ee5f417"} Jul 10 15:08:18.376832 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:18.376439 2572 scope.go:117] "RemoveContainer" containerID="6e56340303aedddc336807ff2543dc47c81f73d7606d3082074ddd3a9688dd53" Jul 10 15:08:18.398747 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:18.398717 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:08:18.404029 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:18.404004 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ff844d/test-model-ff844d-kserve-65b5697489-9rk49"] Jul 10 15:08:19.076690 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:08:19.076654 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7660b1ec-2d05-4d72-b36b-649164d81369" path="/var/lib/kubelet/pods/7660b1ec-2d05-4d72-b36b-649164d81369/volumes" Jul 10 15:09:58.187360 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.187272 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:09:58.187846 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.187668 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7660b1ec-2d05-4d72-b36b-649164d81369" containerName="main" Jul 10 15:09:58.187846 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.187686 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7660b1ec-2d05-4d72-b36b-649164d81369" containerName="main" Jul 10 15:09:58.187846 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.187777 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7660b1ec-2d05-4d72-b36b-649164d81369" containerName="main" Jul 10 15:09:58.190526 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.190510 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.197701 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.197680 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"default-dockercfg-9mkjk\"" Jul 10 15:09:58.197701 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.197695 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"auth-test-model-7b13c8-kserve-self-signed-certs\"" Jul 10 15:09:58.197859 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.197690 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"openshift-service-ca.crt\"" Jul 10 15:09:58.201905 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.201876 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7b13c8\"/\"kube-root-ca.crt\"" Jul 10 15:09:58.214304 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.214270 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:09:58.238019 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.237987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.238019 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.238029 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hdm\" (UniqueName: \"kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.238295 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.238051 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.238295 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.238144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.238295 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.238226 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.238295 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.238253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339702 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339894 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339708 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hdm\" (UniqueName: \"kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339894 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339739 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339894 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339894 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339816 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.339894 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.339840 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.340151 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.340128 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.340219 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.340159 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.340277 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.340251 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.342288 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.342269 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.342453 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.342435 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.353861 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.353832 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hdm\" (UniqueName: \"kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm\") pod \"auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.499578 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.499483 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:09:58.637947 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.637914 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:09:58.641148 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:09:58.641119 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83ed94f1_e544_4ae7_9e74_835d9e96c1a5.slice/crio-db98cfdb75bca9051eccb3b2dc780961292a3c31669930db371eb800d0b71e77 WatchSource:0}: Error finding container db98cfdb75bca9051eccb3b2dc780961292a3c31669930db371eb800d0b71e77: Status 404 returned error can't find the container with id db98cfdb75bca9051eccb3b2dc780961292a3c31669930db371eb800d0b71e77 Jul 10 15:09:58.797956 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.797860 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" event={"ID":"83ed94f1-e544-4ae7-9e74-835d9e96c1a5","Type":"ContainerStarted","Data":"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51"} Jul 10 15:09:58.797956 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.797907 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" event={"ID":"83ed94f1-e544-4ae7-9e74-835d9e96c1a5","Type":"ContainerStarted","Data":"db98cfdb75bca9051eccb3b2dc780961292a3c31669930db371eb800d0b71e77"} Jul 10 15:09:58.820940 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:09:58.820887 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" podStartSLOduration=0.820872558 podStartE2EDuration="820.872558ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:09:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:09:58.819568069 +0000 UTC m=+2088.413923587" watchObservedRunningTime="2026-07-10 15:09:58.820872558 +0000 UTC m=+2088.415228077" Jul 10 15:10:08.500461 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:08.500421 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:10:08.500461 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:08.500477 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:10:08.512990 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:08.512955 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:10:08.855879 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:08.855848 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:10:10.949760 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:10.949733 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:10:10.958205 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:10:10.958173 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:12:15.631034 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:15.631002 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:12:30.651627 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.651559 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" podUID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" containerName="main" containerID="cri-o://c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51" gracePeriod=45 Jul 10 15:12:30.897471 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.897442 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:12:30.978547 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978447 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2hdm\" (UniqueName: \"kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978547 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978505 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978547 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978530 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978547 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978549 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978575 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978595 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs\") pod \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\" (UID: \"83ed94f1-e544-4ae7-9e74-835d9e96c1a5\") " Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978746 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache" (OuterVolumeSpecName: "model-cache") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978813 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home" (OuterVolumeSpecName: "home") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978824 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:12:30.978868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.978864 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-model-cache\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:30.980963 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.980940 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm" (OuterVolumeSpecName: "dshm") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:12:30.980963 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.980954 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm" (OuterVolumeSpecName: "kube-api-access-t2hdm") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "kube-api-access-t2hdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:12:30.981092 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:30.980987 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83ed94f1-e544-4ae7-9e74-835d9e96c1a5" (UID: "83ed94f1-e544-4ae7-9e74-835d9e96c1a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:12:31.079973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.079930 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-home\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:31.079973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.079953 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-dshm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:31.079973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.079962 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tmp-dir\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:31.079973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.079971 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-tls-certs\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:31.079973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.079981 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2hdm\" (UniqueName: \"kubernetes.io/projected/83ed94f1-e544-4ae7-9e74-835d9e96c1a5-kube-api-access-t2hdm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:12:31.448824 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.448789 2572 generic.go:356] "Generic (PLEG): container finished" podID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" containerID="c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51" exitCode=0 Jul 10 15:12:31.448995 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.448830 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" Jul 10 15:12:31.448995 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.448845 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" event={"ID":"83ed94f1-e544-4ae7-9e74-835d9e96c1a5","Type":"ContainerDied","Data":"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51"} Jul 10 15:12:31.448995 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.448878 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv" event={"ID":"83ed94f1-e544-4ae7-9e74-835d9e96c1a5","Type":"ContainerDied","Data":"db98cfdb75bca9051eccb3b2dc780961292a3c31669930db371eb800d0b71e77"} Jul 10 15:12:31.448995 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.448894 2572 scope.go:117] "RemoveContainer" containerID="c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51" Jul 10 15:12:31.457677 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.457657 2572 scope.go:117] "RemoveContainer" containerID="c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51" Jul 10 15:12:31.458009 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:12:31.457989 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51\": container with ID starting with c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51 not found: ID does not exist" containerID="c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51" Jul 10 15:12:31.458057 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.458020 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51"} err="failed to get container status \"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51\": rpc error: code = NotFound desc = could not find container \"c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51\": container with ID starting with c95af99389e9e6b5d7b01d6ea804901dd2064574bbe6a7001bde4fb35cf00c51 not found: ID does not exist" Jul 10 15:12:31.466491 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.466462 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:12:31.470251 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:31.470226 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7b13c8/auth-test-model-7b13c8-kserve-5476ccc5d7-p9vcv"] Jul 10 15:12:33.075737 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:33.075701 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" path="/var/lib/kubelet/pods/83ed94f1-e544-4ae7-9e74-835d9e96c1a5/volumes" Jul 10 15:12:59.887628 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.887588 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:12:59.888088 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.887886 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" containerName="main" Jul 10 15:12:59.888088 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.887896 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" containerName="main" Jul 10 15:12:59.888088 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.887963 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="83ed94f1-e544-4ae7-9e74-835d9e96c1a5" containerName="main" Jul 10 15:12:59.890623 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.890603 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.893536 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.893509 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"openshift-service-ca.crt\"" Jul 10 15:12:59.893843 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.893822 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"kube-root-ca.crt\"" Jul 10 15:12:59.894868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.894848 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"default-dockercfg-kczhc\"" Jul 10 15:12:59.894868 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.894867 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"sub-test-model-e538f1-kserve-self-signed-certs\"" Jul 10 15:12:59.902389 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.902366 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:12:59.934633 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934592 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.934818 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934654 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.934818 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934741 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzmdt\" (UniqueName: \"kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.934818 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934789 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.934981 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934826 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:12:59.934981 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:12:59.934868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.035985 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.035940 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.035985 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.035989 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036285 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036028 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036285 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036055 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzmdt\" (UniqueName: \"kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036285 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036092 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036285 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036493 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036459 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036535 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036517 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.036570 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.036520 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.038630 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.038607 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.038813 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.038794 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.047040 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.047015 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzmdt\" (UniqueName: \"kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt\") pod \"sub-test-model-e538f1-kserve-5f49bc7655-hmqjm\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.200412 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.200327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:00.341077 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.341049 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:13:00.343037 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:13:00.343013 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a572230_7291_4493_a5d5_bcac05277c55.slice/crio-e7bc8d6eb77f9b59ec67203670656500d164575026b0332b8c07013ef16faf70 WatchSource:0}: Error finding container e7bc8d6eb77f9b59ec67203670656500d164575026b0332b8c07013ef16faf70: Status 404 returned error can't find the container with id e7bc8d6eb77f9b59ec67203670656500d164575026b0332b8c07013ef16faf70 Jul 10 15:13:00.569269 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.569154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" event={"ID":"4a572230-7291-4493-a5d5-bcac05277c55","Type":"ContainerStarted","Data":"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5"} Jul 10 15:13:00.569269 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.569208 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" event={"ID":"4a572230-7291-4493-a5d5-bcac05277c55","Type":"ContainerStarted","Data":"e7bc8d6eb77f9b59ec67203670656500d164575026b0332b8c07013ef16faf70"} Jul 10 15:13:00.592053 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:00.591995 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" podStartSLOduration=1.591975928 podStartE2EDuration="1.591975928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:13:00.591508038 +0000 UTC m=+2270.185863558" watchObservedRunningTime="2026-07-10 15:13:00.591975928 +0000 UTC m=+2270.186331447" Jul 10 15:13:10.200746 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:10.200701 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:10.201148 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:10.200765 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:10.213688 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:10.213658 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:10.623862 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:10.623833 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:13:46.226346 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:13:46.226304 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:14:01.246844 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.246733 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" podUID="4a572230-7291-4493-a5d5-bcac05277c55" containerName="main" containerID="cri-o://ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5" gracePeriod=45 Jul 10 15:14:01.498602 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.498533 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:14:01.561427 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561390 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561603 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561454 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561603 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561490 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561603 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561543 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561603 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561598 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzmdt\" (UniqueName: \"kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561825 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561634 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs\") pod \"4a572230-7291-4493-a5d5-bcac05277c55\" (UID: \"4a572230-7291-4493-a5d5-bcac05277c55\") " Jul 10 15:14:01.561825 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561729 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home" (OuterVolumeSpecName: "home") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:14:01.561825 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561781 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:14:01.561825 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561795 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:14:01.562016 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561908 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-home\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.562016 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561921 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-model-cache\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.562016 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.561957 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-tmp-dir\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.564057 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.564018 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm" (OuterVolumeSpecName: "dshm") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:14:01.564186 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.564074 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt" (OuterVolumeSpecName: "kube-api-access-tzmdt") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "kube-api-access-tzmdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:14:01.564186 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.564073 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a572230-7291-4493-a5d5-bcac05277c55" (UID: "4a572230-7291-4493-a5d5-bcac05277c55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:14:01.662716 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.662666 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzmdt\" (UniqueName: \"kubernetes.io/projected/4a572230-7291-4493-a5d5-bcac05277c55-kube-api-access-tzmdt\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.662716 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.662707 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a572230-7291-4493-a5d5-bcac05277c55-tls-certs\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.662929 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.662739 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a572230-7291-4493-a5d5-bcac05277c55-dshm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:14:01.831888 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.831855 2572 generic.go:356] "Generic (PLEG): container finished" podID="4a572230-7291-4493-a5d5-bcac05277c55" containerID="ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5" exitCode=0 Jul 10 15:14:01.832055 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.831905 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" Jul 10 15:14:01.832055 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.831917 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" event={"ID":"4a572230-7291-4493-a5d5-bcac05277c55","Type":"ContainerDied","Data":"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5"} Jul 10 15:14:01.832055 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.831948 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm" event={"ID":"4a572230-7291-4493-a5d5-bcac05277c55","Type":"ContainerDied","Data":"e7bc8d6eb77f9b59ec67203670656500d164575026b0332b8c07013ef16faf70"} Jul 10 15:14:01.832055 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.831965 2572 scope.go:117] "RemoveContainer" containerID="ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5" Jul 10 15:14:01.840820 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.840798 2572 scope.go:117] "RemoveContainer" containerID="ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5" Jul 10 15:14:01.841095 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:14:01.841074 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5\": container with ID starting with ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5 not found: ID does not exist" containerID="ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5" Jul 10 15:14:01.841142 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.841105 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5"} err="failed to get container status \"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5\": rpc error: code = NotFound desc = could not find container \"ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5\": container with ID starting with ef9c31db04b792e3c4acb6b3cb3badd23659e54a6dce0267af6d16c987bd0dc5 not found: ID does not exist" Jul 10 15:14:01.857796 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.857769 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:14:01.871682 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:01.871653 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/sub-test-model-e538f1-kserve-5f49bc7655-hmqjm"] Jul 10 15:14:03.075987 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:03.075952 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a572230-7291-4493-a5d5-bcac05277c55" path="/var/lib/kubelet/pods/4a572230-7291-4493-a5d5-bcac05277c55/volumes" Jul 10 15:14:19.323948 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.323907 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:14:19.324463 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.324230 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a572230-7291-4493-a5d5-bcac05277c55" containerName="main" Jul 10 15:14:19.324463 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.324242 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a572230-7291-4493-a5d5-bcac05277c55" containerName="main" Jul 10 15:14:19.324463 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.324307 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a572230-7291-4493-a5d5-bcac05277c55" containerName="main" Jul 10 15:14:19.328148 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.328114 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.334552 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.334509 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"rate-test-model-e538f1-kserve-self-signed-certs\"" Jul 10 15:14:19.334718 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.334562 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"kube-root-ca.crt\"" Jul 10 15:14:19.334718 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.334593 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"default-dockercfg-kczhc\"" Jul 10 15:14:19.334718 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.334613 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e538f1\"/\"openshift-service-ca.crt\"" Jul 10 15:14:19.337580 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.337557 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:14:19.423785 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.423747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.423984 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.423795 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.423984 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.423833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.423984 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.423868 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.423984 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.423968 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5mcf\" (UniqueName: \"kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.424162 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.424006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.524852 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.524852 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524853 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525103 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524873 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525103 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524894 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525103 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524944 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mcf\" (UniqueName: \"kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525103 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.524963 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525359 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.525324 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525411 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.525359 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.525450 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.525412 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.527373 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.527348 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.527697 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.527680 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.536148 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.536124 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5mcf\" (UniqueName: \"kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf\") pod \"rate-test-model-e538f1-kserve-55ccd94b95-gphtd\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.639763 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.639725 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:19.783813 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.783791 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:14:19.784538 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:14:19.784511 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8428fffa_5cde_4ca4_9c6f_266996998e2c.slice/crio-69f079306daa107fce3483984c3e1d8ead06f0784050179afe370fc290269c50 WatchSource:0}: Error finding container 69f079306daa107fce3483984c3e1d8ead06f0784050179afe370fc290269c50: Status 404 returned error can't find the container with id 69f079306daa107fce3483984c3e1d8ead06f0784050179afe370fc290269c50 Jul 10 15:14:19.911589 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.911497 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" event={"ID":"8428fffa-5cde-4ca4-9c6f-266996998e2c","Type":"ContainerStarted","Data":"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4"} Jul 10 15:14:19.911589 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.911543 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" event={"ID":"8428fffa-5cde-4ca4-9c6f-266996998e2c","Type":"ContainerStarted","Data":"69f079306daa107fce3483984c3e1d8ead06f0784050179afe370fc290269c50"} Jul 10 15:14:19.938763 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:19.938701 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" podStartSLOduration=0.938685034 podStartE2EDuration="938.685034ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:14:19.937554137 +0000 UTC m=+2349.531909660" watchObservedRunningTime="2026-07-10 15:14:19.938685034 +0000 UTC m=+2349.533040553" Jul 10 15:14:29.640570 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:29.640518 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:29.640570 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:29.640578 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:29.653056 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:29.653023 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:14:29.964761 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:14:29.964668 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:15:05.398043 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:05.398005 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:15:06.324145 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.324108 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770257375280"] Jul 10 15:15:06.327666 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.327641 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:06.330713 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.330691 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-wprz9\"" Jul 10 15:15:06.335855 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.335829 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770257375280"] Jul 10 15:15:06.431704 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.431656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l9h5\" (UniqueName: \"kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5\") pod \"test-curl-7741-139770257375280\" (UID: \"54c0b592-d5b7-4f74-9daf-675911027873\") " pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:06.532824 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.532792 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l9h5\" (UniqueName: \"kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5\") pod \"test-curl-7741-139770257375280\" (UID: \"54c0b592-d5b7-4f74-9daf-675911027873\") " pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:06.550826 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.550787 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l9h5\" (UniqueName: \"kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5\") pod \"test-curl-7741-139770257375280\" (UID: \"54c0b592-d5b7-4f74-9daf-675911027873\") " pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:06.637942 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.637909 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:06.773721 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.773583 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770257375280"] Jul 10 15:15:06.785687 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:06.785665 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 15:15:07.114830 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:07.114787 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770257375280" event={"ID":"54c0b592-d5b7-4f74-9daf-675911027873","Type":"ContainerStarted","Data":"478e7776f2eb8b12be56c35d2362cf9be8392520394f9195fc8827414105a9bb"} Jul 10 15:15:08.121305 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:08.121267 2572 generic.go:356] "Generic (PLEG): container finished" podID="54c0b592-d5b7-4f74-9daf-675911027873" containerID="90f0dc29c4e8d674a37374023552567e293f13d8af86febd4ef0b14609672864" exitCode=0 Jul 10 15:15:08.121719 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:08.121364 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770257375280" event={"ID":"54c0b592-d5b7-4f74-9daf-675911027873","Type":"ContainerDied","Data":"90f0dc29c4e8d674a37374023552567e293f13d8af86febd4ef0b14609672864"} Jul 10 15:15:09.262924 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.262899 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:09.301073 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.301041 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770257375280_54c0b592-d5b7-4f74-9daf-675911027873/test-curl-7741-139770257375280/0.log" Jul 10 15:15:09.328209 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.328167 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770257375280"] Jul 10 15:15:09.332510 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.332485 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770257375280"] Jul 10 15:15:09.356169 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.356138 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l9h5\" (UniqueName: \"kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5\") pod \"54c0b592-d5b7-4f74-9daf-675911027873\" (UID: \"54c0b592-d5b7-4f74-9daf-675911027873\") " Jul 10 15:15:09.358513 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.358477 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5" (OuterVolumeSpecName: "kube-api-access-8l9h5") pod "54c0b592-d5b7-4f74-9daf-675911027873" (UID: "54c0b592-d5b7-4f74-9daf-675911027873"). InnerVolumeSpecName "kube-api-access-8l9h5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:09.450337 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.450242 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770254464432"] Jul 10 15:15:09.450640 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.450621 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c0b592-d5b7-4f74-9daf-675911027873" containerName="test-curl-7741-139770257375280" Jul 10 15:15:09.450724 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.450643 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c0b592-d5b7-4f74-9daf-675911027873" containerName="test-curl-7741-139770257375280" Jul 10 15:15:09.450777 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.450751 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c0b592-d5b7-4f74-9daf-675911027873" containerName="test-curl-7741-139770257375280" Jul 10 15:15:09.453528 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.453506 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:09.457540 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.457509 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l9h5\" (UniqueName: \"kubernetes.io/projected/54c0b592-d5b7-4f74-9daf-675911027873-kube-api-access-8l9h5\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:09.460644 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.460623 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770254464432"] Jul 10 15:15:09.558740 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.558704 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t84k\" (UniqueName: \"kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k\") pod \"test-curl-7741-139770254464432\" (UID: \"353a770a-11e3-4966-a95e-8e5f62d9cc36\") " pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:09.659933 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.659871 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t84k\" (UniqueName: \"kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k\") pod \"test-curl-7741-139770254464432\" (UID: \"353a770a-11e3-4966-a95e-8e5f62d9cc36\") " pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:09.668637 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.668606 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t84k\" (UniqueName: \"kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k\") pod \"test-curl-7741-139770254464432\" (UID: \"353a770a-11e3-4966-a95e-8e5f62d9cc36\") " pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:09.763581 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.763492 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:09.896837 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:09.896804 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770254464432"] Jul 10 15:15:09.899664 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:15:09.899638 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod353a770a_11e3_4966_a95e_8e5f62d9cc36.slice/crio-0864637806b28b06f3745eda053ebd24466b9c8292ad3bcd470b6bf4ace2e318 WatchSource:0}: Error finding container 0864637806b28b06f3745eda053ebd24466b9c8292ad3bcd470b6bf4ace2e318: Status 404 returned error can't find the container with id 0864637806b28b06f3745eda053ebd24466b9c8292ad3bcd470b6bf4ace2e318 Jul 10 15:15:10.130551 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:10.130521 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="478e7776f2eb8b12be56c35d2362cf9be8392520394f9195fc8827414105a9bb" Jul 10 15:15:10.130551 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:10.130523 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:10.131724 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:10.131694 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770254464432" event={"ID":"353a770a-11e3-4966-a95e-8e5f62d9cc36","Type":"ContainerStarted","Data":"0864637806b28b06f3745eda053ebd24466b9c8292ad3bcd470b6bf4ace2e318"} Jul 10 15:15:10.141174 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:15:10.141144 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7741-139770257375280\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" podUID="54c0b592-d5b7-4f74-9daf-675911027873" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:10.986843 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:10.986812 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:15:10.990379 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:10.990356 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6ccb_1ba93023-66ea-493f-80b8-ec6119aa9c36/ovn-acl-logging/0.log" Jul 10 15:15:11.074997 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:15:11.074886 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7741-139770257375280\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" podUID="54c0b592-d5b7-4f74-9daf-675911027873" pod="openshift-ingress/test-curl-7741-139770257375280" Jul 10 15:15:11.075964 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:11.075938 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c0b592-d5b7-4f74-9daf-675911027873" path="/var/lib/kubelet/pods/54c0b592-d5b7-4f74-9daf-675911027873/volumes" Jul 10 15:15:11.137766 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:11.137725 2572 generic.go:356] "Generic (PLEG): container finished" podID="353a770a-11e3-4966-a95e-8e5f62d9cc36" containerID="903bf59337309bfa9abb217c7d74e14ae70fb3586b7e292ed3a31f7e14131f25" exitCode=0 Jul 10 15:15:11.137937 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:11.137816 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770254464432" event={"ID":"353a770a-11e3-4966-a95e-8e5f62d9cc36","Type":"ContainerDied","Data":"903bf59337309bfa9abb217c7d74e14ae70fb3586b7e292ed3a31f7e14131f25"} Jul 10 15:15:12.280744 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.280715 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:12.304233 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.304205 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770254464432_353a770a-11e3-4966-a95e-8e5f62d9cc36/test-curl-7741-139770254464432/0.log" Jul 10 15:15:12.337879 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.337839 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770254464432"] Jul 10 15:15:12.344224 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.344176 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770254464432"] Jul 10 15:15:12.384799 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.384765 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t84k\" (UniqueName: \"kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k\") pod \"353a770a-11e3-4966-a95e-8e5f62d9cc36\" (UID: \"353a770a-11e3-4966-a95e-8e5f62d9cc36\") " Jul 10 15:15:12.387007 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.386980 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k" (OuterVolumeSpecName: "kube-api-access-4t84k") pod "353a770a-11e3-4966-a95e-8e5f62d9cc36" (UID: "353a770a-11e3-4966-a95e-8e5f62d9cc36"). InnerVolumeSpecName "kube-api-access-4t84k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:12.486156 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.486061 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t84k\" (UniqueName: \"kubernetes.io/projected/353a770a-11e3-4966-a95e-8e5f62d9cc36-kube-api-access-4t84k\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:12.497684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.497643 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:12.498030 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.498013 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353a770a-11e3-4966-a95e-8e5f62d9cc36" containerName="test-curl-7741-139770254464432" Jul 10 15:15:12.498121 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.498034 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="353a770a-11e3-4966-a95e-8e5f62d9cc36" containerName="test-curl-7741-139770254464432" Jul 10 15:15:12.498121 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.498115 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="353a770a-11e3-4966-a95e-8e5f62d9cc36" containerName="test-curl-7741-139770254464432" Jul 10 15:15:12.501096 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.501074 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:12.509335 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.509306 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:12.688744 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.688707 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbq6f\" (UniqueName: \"kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f\") pod \"test-curl-7741-139770259761024\" (UID: \"393cd13b-42a7-4904-9bb7-fa659b788ec6\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:12.789441 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.789361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbq6f\" (UniqueName: \"kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f\") pod \"test-curl-7741-139770259761024\" (UID: \"393cd13b-42a7-4904-9bb7-fa659b788ec6\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:12.798803 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.798769 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbq6f\" (UniqueName: \"kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f\") pod \"test-curl-7741-139770259761024\" (UID: \"393cd13b-42a7-4904-9bb7-fa659b788ec6\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:12.810664 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.810642 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:12.941937 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:12.941902 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:12.945401 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:15:12.945371 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod393cd13b_42a7_4904_9bb7_fa659b788ec6.slice/crio-3cc13681b022a7cca97a404c7dbc3c50f229ede970980c7c5fa7170b7f28a74e WatchSource:0}: Error finding container 3cc13681b022a7cca97a404c7dbc3c50f229ede970980c7c5fa7170b7f28a74e: Status 404 returned error can't find the container with id 3cc13681b022a7cca97a404c7dbc3c50f229ede970980c7c5fa7170b7f28a74e Jul 10 15:15:13.077013 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:13.076985 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353a770a-11e3-4966-a95e-8e5f62d9cc36" path="/var/lib/kubelet/pods/353a770a-11e3-4966-a95e-8e5f62d9cc36/volumes" Jul 10 15:15:13.148684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:13.148658 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770254464432" Jul 10 15:15:13.148861 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:13.148715 2572 scope.go:117] "RemoveContainer" containerID="903bf59337309bfa9abb217c7d74e14ae70fb3586b7e292ed3a31f7e14131f25" Jul 10 15:15:13.150049 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:13.149884 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259761024" event={"ID":"393cd13b-42a7-4904-9bb7-fa659b788ec6","Type":"ContainerStarted","Data":"3cc13681b022a7cca97a404c7dbc3c50f229ede970980c7c5fa7170b7f28a74e"} Jul 10 15:15:14.157282 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:14.157244 2572 generic.go:356] "Generic (PLEG): container finished" podID="393cd13b-42a7-4904-9bb7-fa659b788ec6" containerID="d7055efd14c3e19f7344d37ed17f164f6060756b24b928da2f9a228cfb8deefa" exitCode=0 Jul 10 15:15:14.157664 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:14.157313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259761024" event={"ID":"393cd13b-42a7-4904-9bb7-fa659b788ec6","Type":"ContainerDied","Data":"d7055efd14c3e19f7344d37ed17f164f6060756b24b928da2f9a228cfb8deefa"} Jul 10 15:15:15.305771 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.305743 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.327526 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.327494 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770259761024_393cd13b-42a7-4904-9bb7-fa659b788ec6/test-curl-7741-139770259761024/0.log" Jul 10 15:15:15.349863 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.349824 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:15.355702 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.355666 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:15.409796 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.409762 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbq6f\" (UniqueName: \"kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f\") pod \"393cd13b-42a7-4904-9bb7-fa659b788ec6\" (UID: \"393cd13b-42a7-4904-9bb7-fa659b788ec6\") " Jul 10 15:15:15.412018 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.411987 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f" (OuterVolumeSpecName: "kube-api-access-hbq6f") pod "393cd13b-42a7-4904-9bb7-fa659b788ec6" (UID: "393cd13b-42a7-4904-9bb7-fa659b788ec6"). InnerVolumeSpecName "kube-api-access-hbq6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:15.477764 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.477724 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:15.478122 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.478104 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="393cd13b-42a7-4904-9bb7-fa659b788ec6" containerName="test-curl-7741-139770259761024" Jul 10 15:15:15.478223 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.478125 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="393cd13b-42a7-4904-9bb7-fa659b788ec6" containerName="test-curl-7741-139770259761024" Jul 10 15:15:15.478223 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.478215 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="393cd13b-42a7-4904-9bb7-fa659b788ec6" containerName="test-curl-7741-139770259761024" Jul 10 15:15:15.481218 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.481175 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.489054 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.489028 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:15.510687 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.510645 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbq6f\" (UniqueName: \"kubernetes.io/projected/393cd13b-42a7-4904-9bb7-fa659b788ec6-kube-api-access-hbq6f\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:15.611669 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.611624 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96t6n\" (UniqueName: \"kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n\") pod \"test-curl-7741-139770259761024\" (UID: \"ee694800-a9bc-49be-b460-6b806ba188be\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.712976 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.712889 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96t6n\" (UniqueName: \"kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n\") pod \"test-curl-7741-139770259761024\" (UID: \"ee694800-a9bc-49be-b460-6b806ba188be\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.722711 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.722677 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96t6n\" (UniqueName: \"kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n\") pod \"test-curl-7741-139770259761024\" (UID: \"ee694800-a9bc-49be-b460-6b806ba188be\") " pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.790055 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.790017 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:15.925899 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:15.925869 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:15.928602 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:15:15.928569 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee694800_a9bc_49be_b460_6b806ba188be.slice/crio-5a5a040545c49c4720772eb51e78400d4fc92239d255f72f804c6c04a42483bf WatchSource:0}: Error finding container 5a5a040545c49c4720772eb51e78400d4fc92239d255f72f804c6c04a42483bf: Status 404 returned error can't find the container with id 5a5a040545c49c4720772eb51e78400d4fc92239d255f72f804c6c04a42483bf Jul 10 15:15:16.166761 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:16.166729 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cc13681b022a7cca97a404c7dbc3c50f229ede970980c7c5fa7170b7f28a74e" Jul 10 15:15:16.166761 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:16.166748 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:16.167919 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:16.167886 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259761024" event={"ID":"ee694800-a9bc-49be-b460-6b806ba188be","Type":"ContainerStarted","Data":"5a5a040545c49c4720772eb51e78400d4fc92239d255f72f804c6c04a42483bf"} Jul 10 15:15:16.179794 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:16.179758 2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7741-139770259761024" oldPodUID="393cd13b-42a7-4904-9bb7-fa659b788ec6" podUID="ee694800-a9bc-49be-b460-6b806ba188be" Jul 10 15:15:17.076389 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:17.076357 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="393cd13b-42a7-4904-9bb7-fa659b788ec6" path="/var/lib/kubelet/pods/393cd13b-42a7-4904-9bb7-fa659b788ec6/volumes" Jul 10 15:15:17.173961 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:17.173926 2572 generic.go:356] "Generic (PLEG): container finished" podID="ee694800-a9bc-49be-b460-6b806ba188be" containerID="0c1543ecddb474264cd70bc78187c6d26de967b6f0b6227434cda2beb5406605" exitCode=0 Jul 10 15:15:17.174143 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:17.174014 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259761024" event={"ID":"ee694800-a9bc-49be-b460-6b806ba188be","Type":"ContainerDied","Data":"0c1543ecddb474264cd70bc78187c6d26de967b6f0b6227434cda2beb5406605"} Jul 10 15:15:18.317594 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.317567 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:18.333063 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.333030 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96t6n\" (UniqueName: \"kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n\") pod \"ee694800-a9bc-49be-b460-6b806ba188be\" (UID: \"ee694800-a9bc-49be-b460-6b806ba188be\") " Jul 10 15:15:18.335599 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.335569 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n" (OuterVolumeSpecName: "kube-api-access-96t6n") pod "ee694800-a9bc-49be-b460-6b806ba188be" (UID: "ee694800-a9bc-49be-b460-6b806ba188be"). InnerVolumeSpecName "kube-api-access-96t6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:18.339886 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.339859 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770259761024_ee694800-a9bc-49be-b460-6b806ba188be/test-curl-7741-139770259761024/0.log" Jul 10 15:15:18.367298 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.367265 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:18.372754 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.372732 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770259761024"] Jul 10 15:15:18.433704 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.433615 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96t6n\" (UniqueName: \"kubernetes.io/projected/ee694800-a9bc-49be-b460-6b806ba188be-kube-api-access-96t6n\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:18.493840 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.493806 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"] Jul 10 15:15:18.494148 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.494133 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee694800-a9bc-49be-b460-6b806ba188be" containerName="test-curl-7741-139770259761024" Jul 10 15:15:18.494213 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.494151 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee694800-a9bc-49be-b460-6b806ba188be" containerName="test-curl-7741-139770259761024" Jul 10 15:15:18.494254 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.494233 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee694800-a9bc-49be-b460-6b806ba188be" containerName="test-curl-7741-139770259761024" Jul 10 15:15:18.497015 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.496999 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:18.504571 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.504541 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"] Jul 10 15:15:18.534489 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.534455 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd7lm\" (UniqueName: \"kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm\") pod \"test-curl-7741-139770259762032\" (UID: \"439e71d8-976b-42ac-a844-f7af598d5593\") " pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:18.635226 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.635162 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd7lm\" (UniqueName: \"kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm\") pod \"test-curl-7741-139770259762032\" (UID: \"439e71d8-976b-42ac-a844-f7af598d5593\") " pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:18.645032 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.645004 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd7lm\" (UniqueName: \"kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm\") pod \"test-curl-7741-139770259762032\" (UID: \"439e71d8-976b-42ac-a844-f7af598d5593\") " pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:18.807741 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.807649 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:18.943484 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:18.943451 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"] Jul 10 15:15:18.946582 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:15:18.946546 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod439e71d8_976b_42ac_a844_f7af598d5593.slice/crio-b116469f163abc8a77e1aed1b718e371a831a939dd492a82f278ad77640fb116 WatchSource:0}: Error finding container b116469f163abc8a77e1aed1b718e371a831a939dd492a82f278ad77640fb116: Status 404 returned error can't find the container with id b116469f163abc8a77e1aed1b718e371a831a939dd492a82f278ad77640fb116 Jul 10 15:15:19.076210 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:19.076115 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee694800-a9bc-49be-b460-6b806ba188be" path="/var/lib/kubelet/pods/ee694800-a9bc-49be-b460-6b806ba188be/volumes" Jul 10 15:15:19.184316 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:19.184288 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259761024" Jul 10 15:15:19.184512 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:19.184328 2572 scope.go:117] "RemoveContainer" containerID="0c1543ecddb474264cd70bc78187c6d26de967b6f0b6227434cda2beb5406605" Jul 10 15:15:19.185668 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:19.185644 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259762032" event={"ID":"439e71d8-976b-42ac-a844-f7af598d5593","Type":"ContainerStarted","Data":"b116469f163abc8a77e1aed1b718e371a831a939dd492a82f278ad77640fb116"} Jul 10 15:15:20.197507 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.197470 2572 generic.go:356] "Generic (PLEG): container finished" podID="439e71d8-976b-42ac-a844-f7af598d5593" containerID="bd0fe836e42d75e571e6345802ed3297a9130fbe47da8a43e4d5b0029d588a00" exitCode=0 Jul 10 15:15:20.197930 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.197545 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7741-139770259762032" event={"ID":"439e71d8-976b-42ac-a844-f7af598d5593","Type":"ContainerDied","Data":"bd0fe836e42d75e571e6345802ed3297a9130fbe47da8a43e4d5b0029d588a00"} Jul 10 15:15:20.420271 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.420183 2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" podUID="8428fffa-5cde-4ca4-9c6f-266996998e2c" containerName="main" containerID="cri-o://1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4" gracePeriod=45 Jul 10 15:15:20.676859 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.676832 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:15:20.749385 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749352 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.749385 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749391 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.749580 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749410 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.749580 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749447 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.749580 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749506 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5mcf\" (UniqueName: \"kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.749773 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749742 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:15:20.749821 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.749788 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home" (OuterVolumeSpecName: "home") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:15:20.751730 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.751696 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:15:20.751730 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.751706 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf" (OuterVolumeSpecName: "kube-api-access-w5mcf") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "kube-api-access-w5mcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:20.752014 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.751763 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm" (OuterVolumeSpecName: "dshm") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:15:20.850415 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850382 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir\") pod \"8428fffa-5cde-4ca4-9c6f-266996998e2c\" (UID: \"8428fffa-5cde-4ca4-9c6f-266996998e2c\") " Jul 10 15:15:20.850602 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850587 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5mcf\" (UniqueName: \"kubernetes.io/projected/8428fffa-5cde-4ca4-9c6f-266996998e2c-kube-api-access-w5mcf\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:20.850684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850605 2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8428fffa-5cde-4ca4-9c6f-266996998e2c-tls-certs\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:20.850684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850616 2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-dshm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:20.850684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850623 2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-model-cache\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:20.850684 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850631 2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-home\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:20.850830 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.850690 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8428fffa-5cde-4ca4-9c6f-266996998e2c" (UID: "8428fffa-5cde-4ca4-9c6f-266996998e2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:15:20.950973 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:20.950935 2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8428fffa-5cde-4ca4-9c6f-266996998e2c-tmp-dir\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:21.202636 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.202545 2572 generic.go:356] "Generic (PLEG): container finished" podID="8428fffa-5cde-4ca4-9c6f-266996998e2c" containerID="1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4" exitCode=0 Jul 10 15:15:21.202636 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.202590 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" Jul 10 15:15:21.202636 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.202626 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" event={"ID":"8428fffa-5cde-4ca4-9c6f-266996998e2c","Type":"ContainerDied","Data":"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4"} Jul 10 15:15:21.203316 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.202665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd" event={"ID":"8428fffa-5cde-4ca4-9c6f-266996998e2c","Type":"ContainerDied","Data":"69f079306daa107fce3483984c3e1d8ead06f0784050179afe370fc290269c50"} Jul 10 15:15:21.203316 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.202683 2572 scope.go:117] "RemoveContainer" containerID="1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4" Jul 10 15:15:21.211991 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.211965 2572 scope.go:117] "RemoveContainer" containerID="1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4" Jul 10 15:15:21.212360 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:15:21.212336 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4\": container with ID starting with 1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4 not found: ID does not exist" containerID="1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4" Jul 10 15:15:21.212434 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.212374 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4"} err="failed to get container status \"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4\": rpc error: code = NotFound desc = could not find container \"1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4\": container with ID starting with 1a046544e60489b1322674bed48f29704753ec1bd70c57bfb04ac2eba951d1f4 not found: ID does not exist" Jul 10 15:15:21.224672 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.224640 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:15:21.227887 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.227855 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e538f1/rate-test-model-e538f1-kserve-55ccd94b95-gphtd"] Jul 10 15:15:21.337914 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.337886 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:21.353587 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.353559 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd7lm\" (UniqueName: \"kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm\") pod \"439e71d8-976b-42ac-a844-f7af598d5593\" (UID: \"439e71d8-976b-42ac-a844-f7af598d5593\") " Jul 10 15:15:21.355571 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.355551 2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm" (OuterVolumeSpecName: "kube-api-access-jd7lm") pod "439e71d8-976b-42ac-a844-f7af598d5593" (UID: "439e71d8-976b-42ac-a844-f7af598d5593"). InnerVolumeSpecName "kube-api-access-jd7lm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:15:21.362655 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.362627 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7741-139770259762032_439e71d8-976b-42ac-a844-f7af598d5593/test-curl-7741-139770259762032/0.log" Jul 10 15:15:21.385910 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.385878 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"] Jul 10 15:15:21.389901 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.389877 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7741-139770259762032"] Jul 10 15:15:21.454272 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:21.454152 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd7lm\" (UniqueName: \"kubernetes.io/projected/439e71d8-976b-42ac-a844-f7af598d5593-kube-api-access-jd7lm\") on node \"ip-10-0-136-92.ec2.internal\" DevicePath \"\"" Jul 10 15:15:22.208279 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:22.208251 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:22.208695 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:22.208284 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b116469f163abc8a77e1aed1b718e371a831a939dd492a82f278ad77640fb116" Jul 10 15:15:22.219006 ip-10-0-136-92 kubenswrapper[2572]: E0710 15:15:22.218978 2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7741-139770259762032\" is forbidden: User \"system:node:ip-10-0-136-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-92.ec2.internal' and this object" podUID="439e71d8-976b-42ac-a844-f7af598d5593" pod="openshift-ingress/test-curl-7741-139770259762032" Jul 10 15:15:23.075895 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:23.075864 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="439e71d8-976b-42ac-a844-f7af598d5593" path="/var/lib/kubelet/pods/439e71d8-976b-42ac-a844-f7af598d5593/volumes" Jul 10 15:15:23.076261 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:23.076248 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8428fffa-5cde-4ca4-9c6f-266996998e2c" path="/var/lib/kubelet/pods/8428fffa-5cde-4ca4-9c6f-266996998e2c/volumes" Jul 10 15:15:39.647309 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:39.647227 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59746674dc-wv6dn_3e93a7a6-2730-4992-88b7-d5663928c8c5/authorino/0.log" Jul 10 15:15:45.237210 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:45.237176 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-t4knc_3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296/copy-manifests/0.log" Jul 10 15:15:45.243940 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:45.243917 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-t4knc_3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296/manager/0.log" Jul 10 15:15:47.269774 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:47.269744 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59746674dc-wv6dn_3e93a7a6-2730-4992-88b7-d5663928c8c5/authorino/0.log" Jul 10 15:15:47.616018 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:47.615990 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dkcfm_a45f9c62-b38b-4c5a-843a-af00b44de9a2/kuadrant-console-plugin/0.log" Jul 10 15:15:48.663789 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:48.663759 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-hvgp4_a80c0148-7b31-468f-a949-6851c247636f/kube-auth-proxy/0.log" Jul 10 15:15:49.123452 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:49.123422 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5rrhf_17654171-fd04-4d6d-851e-23339f4cd211/payload-processing/0.log" Jul 10 15:15:49.683721 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:49.683692 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-m5c6b_81bbce8b-1d31-40b7-899a-b6cb3db446ba/main/0.log" Jul 10 15:15:50.018304 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:50.018225 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vtstz_816f639d-54b2-400d-b92e-efafc9dc27f8/main/0.log" Jul 10 15:15:50.132003 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:50.131977 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-wr5bd_ff06230d-7954-4ed4-ac3f-df9a09a734dd/main/0.log" Jul 10 15:15:57.045312 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:57.045279 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lsnd7_0556a41c-eefb-467e-91bb-209b5bf6d33d/global-pull-secret-syncer/0.log" Jul 10 15:15:57.214091 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:57.214050 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x55kl_9cb7717f-6743-4d0d-b398-1a32ab7bd6e6/konnectivity-agent/0.log" Jul 10 15:15:57.283503 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:15:57.283474 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-92.ec2.internal_9fef107a90282d0eb2837214ea08a54d/haproxy/0.log" Jul 10 15:16:02.365237 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:02.365178 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59746674dc-wv6dn_3e93a7a6-2730-4992-88b7-d5663928c8c5/authorino/0.log" Jul 10 15:16:02.601794 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:02.601765 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dkcfm_a45f9c62-b38b-4c5a-843a-af00b44de9a2/kuadrant-console-plugin/0.log" Jul 10 15:16:05.307625 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.307594 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nm6rr_6317b6ca-23ee-4454-ba74-cef7e7894d32/node-exporter/0.log" Jul 10 15:16:05.329565 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.329538 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nm6rr_6317b6ca-23ee-4454-ba74-cef7e7894d32/kube-rbac-proxy/0.log" Jul 10 15:16:05.351433 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.351406 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nm6rr_6317b6ca-23ee-4454-ba74-cef7e7894d32/init-textfile/0.log" Jul 10 15:16:05.763042 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.763001 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7947b8dbd-6mswc_9888294c-5d6b-4dfc-8394-5cf0866d44fc/telemeter-client/0.log" Jul 10 15:16:05.788444 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.788419 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7947b8dbd-6mswc_9888294c-5d6b-4dfc-8394-5cf0866d44fc/reload/0.log" Jul 10 15:16:05.813308 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:05.813281 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7947b8dbd-6mswc_9888294c-5d6b-4dfc-8394-5cf0866d44fc/kube-rbac-proxy/0.log" Jul 10 15:16:07.580224 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580145 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd"] Jul 10 15:16:07.580610 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580525 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8428fffa-5cde-4ca4-9c6f-266996998e2c" containerName="main" Jul 10 15:16:07.580610 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580538 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8428fffa-5cde-4ca4-9c6f-266996998e2c" containerName="main" Jul 10 15:16:07.580610 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580555 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="439e71d8-976b-42ac-a844-f7af598d5593" containerName="test-curl-7741-139770259762032" Jul 10 15:16:07.580610 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580561 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="439e71d8-976b-42ac-a844-f7af598d5593" containerName="test-curl-7741-139770259762032" Jul 10 15:16:07.580610 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580612 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="439e71d8-976b-42ac-a844-f7af598d5593" containerName="test-curl-7741-139770259762032" Jul 10 15:16:07.580765 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.580621 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8428fffa-5cde-4ca4-9c6f-266996998e2c" containerName="main" Jul 10 15:16:07.583769 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.583747 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.586863 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.586844 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c6l7l\"/\"default-dockercfg-qchrr\"" Jul 10 15:16:07.586984 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.586891 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6l7l\"/\"kube-root-ca.crt\"" Jul 10 15:16:07.587883 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.587863 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6l7l\"/\"openshift-service-ca.crt\"" Jul 10 15:16:07.593994 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.593966 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd"] Jul 10 15:16:07.746649 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.746606 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-podres\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.746649 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.746658 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-proc\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.746875 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.746680 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-sys\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.746875 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.746776 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5shd\" (UniqueName: \"kubernetes.io/projected/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-kube-api-access-g5shd\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.746875 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.746821 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-lib-modules\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847688 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847576 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5shd\" (UniqueName: \"kubernetes.io/projected/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-kube-api-access-g5shd\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847688 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847634 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-lib-modules\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847752 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-podres\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847797 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-proc\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847807 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-podres\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847757 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-lib-modules\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847833 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-sys\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.847927 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847860 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-proc\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.848150 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.847929 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-sys\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.856781 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.856754 2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5shd\" (UniqueName: \"kubernetes.io/projected/e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5-kube-api-access-g5shd\") pod \"perf-node-gather-daemonset-97nvd\" (UID: \"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5\") " pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:07.894498 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:07.894458 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:08.035853 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:08.035777 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd"] Jul 10 15:16:08.038556 ip-10-0-136-92 kubenswrapper[2572]: W0710 15:16:08.038527 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode7c56bba_fdbe_4a01_a9e1_4aaf1eb196b5.slice/crio-45e889561f1d5481ec800abda28372887a6798e9a1afe6777d14736866f4e912 WatchSource:0}: Error finding container 45e889561f1d5481ec800abda28372887a6798e9a1afe6777d14736866f4e912: Status 404 returned error can't find the container with id 45e889561f1d5481ec800abda28372887a6798e9a1afe6777d14736866f4e912 Jul 10 15:16:08.411547 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:08.411505 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" event={"ID":"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5","Type":"ContainerStarted","Data":"80ae7a67a8be1d8b781aa1f68d566a67e8c0862a8c286e59f80e871851a5a823"} Jul 10 15:16:08.411742 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:08.411566 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" event={"ID":"e7c56bba-fdbe-4a01-a9e1-4aaf1eb196b5","Type":"ContainerStarted","Data":"45e889561f1d5481ec800abda28372887a6798e9a1afe6777d14736866f4e912"} Jul 10 15:16:08.411742 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:08.411595 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:08.449919 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:08.449812 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" podStartSLOduration=1.449794764 podStartE2EDuration="1.449794764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:16:08.449558126 +0000 UTC m=+2458.043913645" watchObservedRunningTime="2026-07-10 15:16:08.449794764 +0000 UTC m=+2458.044150280" Jul 10 15:16:10.205657 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:10.205627 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t6tmh_99eeec30-4f73-403f-93e7-db1d9e4a1a8a/dns/0.log" Jul 10 15:16:10.228613 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:10.228586 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-t6tmh_99eeec30-4f73-403f-93e7-db1d9e4a1a8a/kube-rbac-proxy/0.log" Jul 10 15:16:10.324720 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:10.324696 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-txdwq_b224fd08-5b50-4b09-8c9c-30e22d27b052/dns-node-resolver/0.log" Jul 10 15:16:10.893913 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:10.893873 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-b595bc954-tfnt2_cea3443d-16f0-4e68-86f8-ad49b705d64b/registry/0.log" Jul 10 15:16:10.970809 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:10.970783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r6trl_57c89553-a981-4c32-8289-e82919822315/node-ca/0.log" Jul 10 15:16:12.173170 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:12.173118 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-hvgp4_a80c0148-7b31-468f-a949-6851c247636f/kube-auth-proxy/0.log" Jul 10 15:16:12.377011 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:12.376955 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5rrhf_17654171-fd04-4d6d-851e-23339f4cd211/payload-processing/0.log" Jul 10 15:16:13.133713 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:13.133669 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q2zdk_7a9ad40e-433f-4810-b306-802e5366572f/serve-healthcheck-canary/0.log" Jul 10 15:16:13.855842 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:13.855810 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4p47_31f13d04-41b9-4251-bfef-d446f9e4eee8/kube-rbac-proxy/0.log" Jul 10 15:16:13.881306 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:13.881280 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4p47_31f13d04-41b9-4251-bfef-d446f9e4eee8/exporter/0.log" Jul 10 15:16:13.905294 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:13.905248 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z4p47_31f13d04-41b9-4251-bfef-d446f9e4eee8/extractor/0.log" Jul 10 15:16:14.426666 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:14.426639 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c6l7l/perf-node-gather-daemonset-97nvd" Jul 10 15:16:16.914500 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:16.914454 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-t4knc_3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296/manager/0.log" Jul 10 15:16:16.939872 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:16.939845 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85d85459f4-t4knc_3a9d95ae-01ec-4cd4-b4cb-4dacd13b0296/copy-manifests/0.log" Jul 10 15:16:18.096705 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:18.096663 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-hvgp4_a80c0148-7b31-468f-a949-6851c247636f/kube-auth-proxy/0.log" Jul 10 15:16:18.238858 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:18.238827 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5rrhf_17654171-fd04-4d6d-851e-23339f4cd211/payload-processing/0.log" Jul 10 15:16:18.304919 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:18.304880 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-869f49798c-vvj7c_32145e5f-b3cc-40b9-b055-684bee700c51/manager/0.log" Jul 10 15:16:19.713363 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:19.713330 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74544d9668-hvgp4_a80c0148-7b31-468f-a949-6851c247636f/kube-auth-proxy/0.log" Jul 10 15:16:19.874908 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:19.874878 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5rrhf_17654171-fd04-4d6d-851e-23339f4cd211/payload-processing/0.log" Jul 10 15:16:23.598808 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:23.598777 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-crz89_6db94d02-685e-4d88-aca9-d5e4eead3481/migrator/0.log" Jul 10 15:16:23.620402 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:23.620370 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-crz89_6db94d02-685e-4d88-aca9-d5e4eead3481/graceful-termination/0.log" Jul 10 15:16:25.189022 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:25.188997 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6hkm_97c0fe49-1932-48b3-9f6a-0b4b0d0844fd/kube-multus-additional-cni-plugins/0.log" Jul 10 15:16:25.214960 ip-10-0-136-92 kubenswrapper[2572]: I0710 15:16:25.214930 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n6hkm_97c0fe49-1932-48b3-9f6a-0b4b0d0844fd/egress-router-binary-copy/0.log"