Jul 14 09:24:54.530587 ip-10-0-129-15 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 14 09:24:54.530596 ip-10-0-129-15 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 14 09:24:54.530603 ip-10-0-129-15 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 14 09:24:54.530818 ip-10-0-129-15 systemd[1]: Failed to start Kubernetes Kubelet. Jul 14 09:25:04.587610 ip-10-0-129-15 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 14 09:25:04.587626 ip-10-0-129-15 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e0a43da8d23347c39e1a3f4c2139153e -- Jul 14 09:26:41.403494 ip-10-0-129-15 systemd[1]: Starting Kubernetes Kubelet... Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.811007 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816876 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816889 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816892 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816895 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816898 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816902 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816904 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816907 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.143829 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816909 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:41.883715 ip-10-0-129-15 systemd[1]: Started Kubernetes Kubelet. Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816912 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816915 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816918 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816920 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816923 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816926 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816928 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816932 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816936 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816939 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816942 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816947 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816955 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816958 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816961 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816964 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816967 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816969 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.152702 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816972 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816974 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816977 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816980 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816982 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816984 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816987 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816990 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816992 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816995 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.816997 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817000 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817002 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817005 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817007 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817010 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817012 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817015 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817017 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817020 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.153462 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817022 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817025 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817028 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817030 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817033 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817036 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817038 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817042 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817045 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817048 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817051 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817054 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817056 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817059 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817062 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817065 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817067 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817070 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817072 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817075 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.154691 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817078 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817081 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817083 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817086 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817088 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817091 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817094 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817097 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817099 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817102 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817105 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817110 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817113 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817116 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817119 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817122 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817124 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817127 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817129 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.155831 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817132 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817135 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817137 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817140 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817142 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817145 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817148 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817151 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817153 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817156 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817159 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817161 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817164 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817861 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817867 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817870 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817873 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817876 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817879 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817881 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.158062 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817884 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817887 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817889 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817892 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817894 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817896 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817899 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817902 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817905 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817908 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817911 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817926 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817930 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817933 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.817977 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818009 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818012 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818016 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818018 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.160306 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818021 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818025 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818028 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818031 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818034 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818037 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818043 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818051 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818054 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818058 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818063 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818066 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818069 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818072 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818075 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818078 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818081 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818088 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.163860 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.818091 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819410 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819444 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819451 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819556 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819563 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819567 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819569 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819576 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819581 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819584 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819587 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819590 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819593 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819596 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819599 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819602 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819605 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819608 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819610 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.166625 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819613 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819616 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819619 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819621 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819624 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819626 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819629 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819631 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819635 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819637 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819640 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819642 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819646 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819648 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819650 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819654 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819656 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819659 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819662 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819665 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.169165 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819668 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819670 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819673 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819676 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819679 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819682 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819684 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819687 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819690 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819693 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819695 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819698 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819700 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819703 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.819707 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820134 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820140 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820143 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820146 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820149 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.170448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820152 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820155 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820158 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820160 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820163 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820165 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820167 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820171 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820174 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820176 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820196 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820199 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820202 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820204 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820207 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820210 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820213 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820215 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820218 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.172534 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820222 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820226 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820229 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820232 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820236 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820239 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820242 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820246 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820250 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820252 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820256 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820259 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820261 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820271 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820274 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820277 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820279 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820282 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820284 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.174357 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820287 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820289 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820292 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820294 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820297 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820299 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820303 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820307 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820309 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820312 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820314 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820317 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820320 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820322 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820324 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820327 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820329 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820333 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820336 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820338 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.175410 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820341 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820343 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820346 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820348 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820351 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820353 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820356 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820364 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820367 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820369 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820372 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820374 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820377 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820379 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820382 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820384 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820389 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820393 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820398 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820401 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.176492 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820403 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820406 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820409 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820411 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820414 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820417 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820419 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820422 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820424 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820428 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820431 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820433 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820436 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820438 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820441 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.820443 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820509 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820515 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820524 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820528 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820545 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 09:26:42.177423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820549 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820553 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820557 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820560 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820563 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820566 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820569 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820572 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820575 2576 flags.go:64] FLAG: --cgroup-root="" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820579 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820582 2576 flags.go:64] FLAG: --client-ca-file="" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820584 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820588 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820595 2576 flags.go:64] FLAG: --cluster-domain="" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820598 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820601 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820604 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820607 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820611 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820614 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820618 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820621 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820624 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820627 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 09:26:42.178361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820630 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820633 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820636 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820640 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820643 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820645 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820648 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820651 2576 flags.go:64] FLAG: --enable-server="true" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820660 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820666 2576 flags.go:64] FLAG: --event-burst="100" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820670 2576 flags.go:64] FLAG: --event-qps="50" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820672 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820675 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820678 2576 flags.go:64] FLAG: --eviction-hard="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820681 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820684 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820687 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820692 2576 flags.go:64] FLAG: --eviction-soft="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820695 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820699 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820702 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820705 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820707 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820710 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820713 2576 flags.go:64] FLAG: --feature-gates="" Jul 14 09:26:42.179321 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820716 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820719 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820722 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820727 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820730 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820733 2576 flags.go:64] FLAG: --help="false" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820736 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820739 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820742 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820745 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820748 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820751 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820754 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820756 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820759 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820762 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820772 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820776 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820778 2576 flags.go:64] FLAG: --kube-reserved="" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820781 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820784 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820787 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820789 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820792 2576 flags.go:64] FLAG: --lock-file="" Jul 14 09:26:42.180549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820796 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820799 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820801 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820811 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820814 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820816 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820819 2576 flags.go:64] FLAG: --logging-format="text" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820821 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820825 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820827 2576 flags.go:64] FLAG: --manifest-url="" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820830 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820835 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820839 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820843 2576 flags.go:64] FLAG: --max-pods="110" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820845 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820848 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820851 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820854 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820857 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820860 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820863 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820869 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820872 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820875 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 09:26:42.181616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820878 2576 flags.go:64] FLAG: --pod-cidr="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820888 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820894 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820896 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820899 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820902 2576 flags.go:64] FLAG: --port="10250" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820905 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820908 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0152127c9c1e37598" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820912 2576 flags.go:64] FLAG: --qos-reserved="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820915 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820918 2576 flags.go:64] FLAG: --register-node="true" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820921 2576 flags.go:64] FLAG: --register-with-taints="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820924 2576 flags.go:64] FLAG: --registry-burst="10" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820927 2576 flags.go:64] FLAG: --registry-qps="5" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820930 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820932 2576 flags.go:64] FLAG: --reserved-memory="" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820936 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820939 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820942 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820945 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820949 2576 flags.go:64] FLAG: --runonce="false" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820951 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820954 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820957 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820960 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 09:26:42.186474 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820963 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820966 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820969 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820972 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820974 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820977 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820980 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820983 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820986 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820994 2576 flags.go:64] FLAG: --system-cgroups="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.820997 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821002 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821005 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821007 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821014 2576 flags.go:64] FLAG: --tls-min-version="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821018 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821020 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821023 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821026 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821028 2576 flags.go:64] FLAG: --v="2" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821032 2576 flags.go:64] FLAG: --version="false" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821036 2576 flags.go:64] FLAG: --vmodule="" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821039 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821043 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821160 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.187687 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821164 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821167 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821171 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821174 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821177 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821191 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821195 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821198 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821200 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821203 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821206 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821208 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821212 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821215 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821218 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821222 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821225 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821234 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821237 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.195605 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821240 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821243 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821246 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821250 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821253 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821257 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821259 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821262 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821265 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821267 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821270 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821273 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821276 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821279 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821282 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821286 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821290 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821292 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821295 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.196579 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821297 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821300 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821302 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821305 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821307 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821310 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821312 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821315 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821318 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821321 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821323 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821326 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821334 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821337 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821339 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821342 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821345 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821348 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821350 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821353 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.197535 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821355 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821357 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821360 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821362 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821365 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821367 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821370 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821372 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821376 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821378 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821381 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821383 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821386 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821388 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821391 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821393 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821396 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821398 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821401 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821404 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.198521 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821406 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821409 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821411 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821414 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821416 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821424 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821427 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821430 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821434 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821436 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821439 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821441 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821444 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821446 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821449 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821451 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821453 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821457 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821460 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.199485 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.821463 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.821948 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.828493 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.828510 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.828523 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.828529 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828583 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828587 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828591 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828594 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828597 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828600 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828603 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828606 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828609 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.200448 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828611 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828614 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828617 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828620 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828623 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828625 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828628 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828630 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828632 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828635 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828638 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828640 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828643 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828645 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828650 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828654 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828658 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828661 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828664 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.201169 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828667 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828671 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828674 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828676 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828681 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828684 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828687 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828690 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828692 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828695 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828697 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828700 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828702 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828705 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828707 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828710 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828713 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828716 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828718 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828721 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.202135 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828723 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828726 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828728 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828731 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828733 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828736 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828738 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828741 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828743 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828746 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828748 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828751 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828754 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828758 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828762 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828765 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828769 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828771 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828774 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828777 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.203035 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828779 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828781 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828784 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828787 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828789 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828792 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828794 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828797 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828799 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828802 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828804 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828807 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828809 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828812 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828814 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828817 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828819 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828822 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828824 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.257807 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828827 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828829 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828832 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828834 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828837 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828840 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828843 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828846 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828849 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828854 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828857 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828859 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.828864 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828976 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828980 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 09:26:42.258767 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828983 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828986 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828989 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828992 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828995 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.828997 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829000 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829003 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829005 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829008 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829010 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829013 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829016 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829018 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829020 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829023 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829025 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829028 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829030 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829032 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 09:26:42.259479 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829035 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829037 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829040 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829044 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829047 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829049 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829052 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829056 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829059 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829062 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829064 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829067 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829069 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829072 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829074 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829077 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829080 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829082 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829085 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 09:26:42.260374 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829087 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829090 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829093 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829095 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829098 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829100 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829103 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829105 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829108 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829110 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829113 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829116 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829120 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829123 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829126 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829129 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829132 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829136 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829140 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829142 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 09:26:42.261493 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829148 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829151 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829153 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829157 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829159 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829162 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829164 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829167 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829170 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829173 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829177 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829192 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829196 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829198 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829202 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829204 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829207 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829209 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829212 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829215 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 09:26:42.262407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829217 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829220 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829222 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829225 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829227 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829230 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829233 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829235 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829239 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829242 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829244 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829247 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829251 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829254 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829257 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829259 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829262 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 09:26:42.263544 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:41.829264 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.829269 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.829377 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.831363 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.832702 2576 server.go:1019] "Starting client certificate rotation" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.832798 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.832836 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.835372 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.836631 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.848495 2576 log.go:25] "Validated CRI v1 runtime API" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.851702 2576 log.go:25] "Validated CRI v1 image API" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.853379 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 09:26:42.264293 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.858904 2576 fs.go:135] Filesystem UUIDs: map[5b58c4df-2d0c-4394-8c6c-dbfaeb319e9e:/dev/nvme0n1p4 7482c77e-c55c-4ec6-9f51-92e68a380485:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 14 09:26:42.265014 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.858917 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.864860 2576 manager.go:216] Machine: {Timestamp:2026-07-14 09:26:41.862752777 +0000 UTC m=+0.340788191 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123463 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cf5d431ab686437c4bd6d26080a05 SystemUUID:ec2cf5d4-31ab-6864-37c4-bd6d26080a05 BootID:e0a43da8-d233-47c3-9e1a-3f4c2139153e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:f2:7d:c7:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:f2:7d:c7:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:aa:bc:26:e2:03 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.864956 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.865029 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.868027 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.868235 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.868271 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-15.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 09:26:42.265483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.869126 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.869136 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.869144 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.869890 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.870697 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873516 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873644 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873665 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873675 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873684 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873694 2576 kubelet.go:398] "Adding apiserver pod source" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873702 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873981 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.873996 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.876371 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.877838 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.877865 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878579 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878593 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878599 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878606 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878616 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878622 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878759 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878765 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878771 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878777 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878786 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.878796 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.879710 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.879721 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.882891 2576 server.go:1268] "Started kubelet" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.882992 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.883044 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.883109 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.884400 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.885122 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.885391 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.887361 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-15.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 09:26:42.265972 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.889724 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-15.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.890938 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.892865 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.892899 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.893041 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.893622 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.893625 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.893654 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.893783 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.893791 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.889885 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-15.ec2.internal.18c21dd4b9fe23e9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-15.ec2.internal,UID:ip-10-0-129-15.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-15.ec2.internal,},FirstTimestamp:2026-07-14 09:26:41.882850281 +0000 UTC m=+0.360885696,LastTimestamp:2026-07-14 09:26:41.882850281 +0000 UTC m=+0.360885696,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-15.ec2.internal,}" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.895514 2576 factory.go:55] Registering systemd factory Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.895531 2576 factory.go:223] Registration of the systemd container factory successfully Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896062 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v9tmg" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896097 2576 factory.go:153] Registering CRI-O factory Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896106 2576 factory.go:223] Registration of the crio container factory successfully Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896146 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896170 2576 factory.go:103] Registering Raw factory Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896194 2576 manager.go:1196] Started watching for new ooms in manager Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.896697 2576 manager.go:349] Starting recovery of all containers Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.897678 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-15.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.897842 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.907351 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v9tmg" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.910255 2576 manager.go:354] Recovery completed Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.914852 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.916541 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.916565 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.916575 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917010 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917014 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917017 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917049 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 14 09:26:42.269938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917680 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.917708 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.918114 2576 policy_none.go:49] "None policy: Start" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.918123 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.918125 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.918276 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.918689 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-15.ec2.internal.18c21dd4bc006a24 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-15.ec2.internal,UID:ip-10-0-129-15.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-15.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-15.ec2.internal,},FirstTimestamp:2026-07-14 09:26:41.916553764 +0000 UTC m=+0.394589186,LastTimestamp:2026-07-14 09:26:41.916553764 +0000 UTC m=+0.394589186,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-15.ec2.internal,}" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.919646 2576 policy_none.go:47] "Start" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.975723 2576 manager.go:337] "Starting Device Plugin manager" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.975838 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.975847 2576 server.go:87] "Starting device plugin registration server" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.976477 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.976495 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.976989 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.977058 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.977066 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.979218 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:41.979277 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:41.993673 2576 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2607 into the system container \"/system.slice\": " Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.077232 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.078007 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.078032 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.078045 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.078080 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.086995 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.087013 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-15.ec2.internal\": node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.102295 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.271627 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.202671 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.336509 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.303126 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.403983 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.403819 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.434761 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.435881 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.435916 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.435938 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.435973 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 09:26:42.478665 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.438635 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.549248 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.549247 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal"] Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.549325 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.550749 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.550779 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.550825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.550791 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.552737 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552088 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.552737 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.552737 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552293 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552933 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552965 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552988 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552991 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.552998 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.553457 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.553007 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.554392 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.554088 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.554392 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.554114 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 09:26:42.555561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.554816 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientMemory" Jul 14 09:26:42.555561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.554845 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 09:26:42.555561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.554859 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeHasSufficientPID" Jul 14 09:26:42.621745 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.596796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.621745 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.596820 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.621745 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.596844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/44c9368520183df5fbab72347490a409-config\") pod \"kube-apiserver-proxy-ip-10-0-129-15.ec2.internal\" (UID: \"44c9368520183df5fbab72347490a409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.621745 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.616305 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-15.ec2.internal\" not found" node="ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.621745 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.621239 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-15.ec2.internal\" not found" node="ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.650397 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697574 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/44c9368520183df5fbab72347490a409-config\") pod \"kube-apiserver-proxy-ip-10-0-129-15.ec2.internal\" (UID: \"44c9368520183df5fbab72347490a409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697626 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/44c9368520183df5fbab72347490a409-config\") pod \"kube-apiserver-proxy-ip-10-0-129-15.ec2.internal\" (UID: \"44c9368520183df5fbab72347490a409\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697631 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.697648 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7d00eeede512e39bcca5885b1a3feaa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal\" (UID: \"f7d00eeede512e39bcca5885b1a3feaa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.750538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.721075 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:42.752127 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.751060 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.892825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.832532 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 09:26:42.892825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.832632 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 09:26:42.892825 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.832672 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 09:26:42.892825 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.851814 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.911579 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 09:21:42 +0000 UTC" deadline="2028-02-27 21:34:09.867434121 +0000 UTC" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.911603 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14244h7m26.9558344s" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.919739 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.923936 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:42.952194 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:42.970935 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:42.969813 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:43.053367 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.052845 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:43.186877 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.152907 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:43.253788 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.253754 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-15.ec2.internal\" not found" Jul 14 09:26:43.326072 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.326014 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:43.394327 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.394301 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" Jul 14 09:26:43.410471 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.410448 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 09:26:43.411755 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.411739 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" Jul 14 09:26:43.419019 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.419005 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 09:26:43.684948 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.684882 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:43.875418 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.875396 2576 apiserver.go:52] "Watching apiserver" Jul 14 09:26:43.884986 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.884961 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 09:26:43.886330 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.886307 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-qh6lf","openshift-network-diagnostics/network-check-target-kcts2","kube-system/global-pull-secret-syncer-s2jnc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n","openshift-dns/node-resolver-pw8nq","openshift-multus/multus-76qwn","openshift-multus/multus-additional-cni-plugins-gk9tm","openshift-network-operator/iptables-alerter-vnvfh","openshift-ovn-kubernetes/ovnkube-node-v7dl8","kube-system/konnectivity-agent-pwxws","kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal","openshift-cluster-node-tuning-operator/tuned-h46bl","openshift-image-registry/node-ca-5tkgk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal"] Jul 14 09:26:43.887789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.887766 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.889706 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.889662 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:43.890706 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.890635 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.890852 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.890824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:43.891116 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.891003 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:43.891240 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.891220 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 09:26:43.891240 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.891231 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 09:26:43.891359 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.891242 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.891493 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.891477 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-68859\"" Jul 14 09:26:43.891959 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.891937 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 09:26:43.892166 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.892150 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 09:26:43.892663 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.892646 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9wzq8\"" Jul 14 09:26:43.892758 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.892707 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.892758 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.892707 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.892865 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.892836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.894018 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.893993 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 09:26:43.894123 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.894072 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:43.894995 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.894974 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.895175 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.895148 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 09:26:43.895335 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.895227 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sc9bh\"" Jul 14 09:26:43.895335 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.895265 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.895512 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.895494 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-76qwn" Jul 14 09:26:43.896679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.896664 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.896798 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.896783 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.896907 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.896844 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-45wtk\"" Jul 14 09:26:43.897358 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.897341 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:43.897961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.897938 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 09:26:43.898055 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.898042 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-98x9m\"" Jul 14 09:26:43.899008 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.898989 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:43.899086 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:43.899018 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c9368520183df5fbab72347490a409.slice/crio-a04662e4b305f9c355cfc78d8058d97c5432050a1dfd2928bc37a8d00635a581 WatchSource:0}: Error finding container a04662e4b305f9c355cfc78d8058d97c5432050a1dfd2928bc37a8d00635a581: Status 404 returned error can't find the container with id a04662e4b305f9c355cfc78d8058d97c5432050a1dfd2928bc37a8d00635a581 Jul 14 09:26:43.899131 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.899087 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:43.899892 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.899872 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 09:26:43.900018 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.900002 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.900198 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.900159 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wmlhg\"" Jul 14 09:26:43.900470 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.900161 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.900578 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.900431 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:43.900578 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:43.900550 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:43.901081 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.901062 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 09:26:43.901137 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.901098 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 09:26:43.901137 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.901099 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 09:26:43.902337 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.902319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:43.902627 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:43.902604 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d00eeede512e39bcca5885b1a3feaa.slice/crio-22ccd2740ca104c5d4a793d3de015691d57e29e2f4b6a2029b14db98cf7e03a5 WatchSource:0}: Error finding container 22ccd2740ca104c5d4a793d3de015691d57e29e2f4b6a2029b14db98cf7e03a5: Status 404 returned error can't find the container with id 22ccd2740ca104c5d4a793d3de015691d57e29e2f4b6a2029b14db98cf7e03a5 Jul 14 09:26:43.903866 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.903849 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:43.904788 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.904768 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 09:26:43.905073 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905054 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7h9hb\"" Jul 14 09:26:43.905073 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905065 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 09:26:43.905230 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905230 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905108 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905230 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905157 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-system-cni-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905230 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-binary-copy\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-dbus\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:43.905534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905323 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9fmc\" (UniqueName: \"kubernetes.io/projected/088c7d44-3d7c-4573-8850-8af99b26bb54-kube-api-access-f9fmc\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905573 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-device-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905604 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905626 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dead7e1a-456c-4dc9-aba6-799c6b8c945f-hosts-file\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dead7e1a-456c-4dc9-aba6-799c6b8c945f-tmp-dir\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.905761 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ed79517-5b5e-4b46-ab41-a0f2831fe646-host\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-kubelet-config\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905836 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62ns\" (UniqueName: \"kubernetes.io/projected/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kube-api-access-b62ns\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905861 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbjj6\" (UniqueName: \"kubernetes.io/projected/dead7e1a-456c-4dc9-aba6-799c6b8c945f-kube-api-access-kbjj6\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905905 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-cnibin\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905925 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-os-release\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905947 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6ed79517-5b5e-4b46-ab41-a0f2831fe646-serviceca\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.905984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzsb6\" (UniqueName: \"kubernetes.io/projected/6ed79517-5b5e-4b46-ab41-a0f2831fe646-kube-api-access-zzsb6\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906102 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:43.906449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906256 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:26:43.906974 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906463 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.906974 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906546 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.906974 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906710 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 09:26:43.906974 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.906746 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86rnb\"" Jul 14 09:26:43.908096 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.908074 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 09:26:43.908963 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.908930 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzgp7\"" Jul 14 09:26:43.909073 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.909046 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 09:26:43.909234 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.909217 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:26:43.935071 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.935026 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2dv2" Jul 14 09:26:43.946086 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.946069 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v2dv2" Jul 14 09:26:43.994750 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:43.994727 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 09:26:44.006592 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqbq\" (UniqueName: \"kubernetes.io/projected/328b120f-2c8a-4462-b843-47ad0e7c9d92-kube-api-access-8lqbq\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.006692 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw9mh\" (UniqueName: \"kubernetes.io/projected/17a11154-32e1-4161-b43f-26881d05f427-kube-api-access-rw9mh\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.006692 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006620 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.006692 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006645 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.006846 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006702 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.006846 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006762 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.006846 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006800 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-kubernetes\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.006846 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006828 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-sys\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-tuned\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006883 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-dbus\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006910 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006941 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.006995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:44.007044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-bin\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-lib-modules\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007060 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007083 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-dbus\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-var-lib-kubelet\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007134 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-systemd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007160 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-cnibin\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007186 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6ed79517-5b5e-4b46-ab41-a0f2831fe646-serviceca\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007233 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-cnibin\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzsb6\" (UniqueName: \"kubernetes.io/projected/6ed79517-5b5e-4b46-ab41-a0f2831fe646-kube-api-access-zzsb6\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-netns\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-netns\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007322 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-env-overrides\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007359 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-modprobe-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.007420 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007388 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-etc-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-system-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007466 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2rnh\" (UniqueName: \"kubernetes.io/projected/bcd140d7-76f7-4094-a426-093973d47036-kube-api-access-b2rnh\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-netd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007526 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-device-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007618 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-device-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007604 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-k8s-cni-cncf-io\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007646 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-kubelet\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-ovn\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007676 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6ed79517-5b5e-4b46-ab41-a0f2831fe646-serviceca\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sslck\" (UniqueName: \"kubernetes.io/projected/bc0962d2-8ea1-4296-8e39-a4d59d12d652-kube-api-access-sslck\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.008022 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ed79517-5b5e-4b46-ab41-a0f2831fe646-host\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007847 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-cni-binary-copy\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-socket-dir-parent\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007879 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-multus-daemon-config\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007901 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-kubelet\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007917 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17a11154-32e1-4161-b43f-26881d05f427-host-slash\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007846 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ed79517-5b5e-4b46-ab41-a0f2831fe646-host\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysconfig\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-conf-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.007986 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-multus-certs\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008006 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-systemd-units\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-script-lib\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008057 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-host\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.008567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008103 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-bin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14481b43-9d55-419e-9072-e2bb2e0f54d8-konnectivity-ca\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-slash\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008151 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-systemd\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-run\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9fmc\" (UniqueName: \"kubernetes.io/projected/088c7d44-3d7c-4573-8850-8af99b26bb54-kube-api-access-f9fmc\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dead7e1a-456c-4dc9-aba6-799c6b8c945f-tmp-dir\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbjj6\" (UniqueName: \"kubernetes.io/projected/dead7e1a-456c-4dc9-aba6-799c6b8c945f-kube-api-access-kbjj6\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-cnibin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008336 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-conf\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008350 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-tmp\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008366 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-log-socket\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008382 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovn-node-metrics-cert\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.009286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tf6p\" (UniqueName: \"kubernetes.io/projected/548dabe2-0b7f-42a2-9f36-833d0943b948-kube-api-access-5tf6p\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008435 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-var-lib-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008486 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-os-release\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-os-release\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008571 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-system-cni-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-binary-copy\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008636 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-os-release\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008681 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-system-cni-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008700 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-multus\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008713 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dead7e1a-456c-4dc9-aba6-799c6b8c945f-tmp-dir\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.009747 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008733 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-hostroot\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-etc-kubernetes\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17a11154-32e1-4161-b43f-26881d05f427-iptables-alerter-script\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008851 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008876 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dead7e1a-456c-4dc9-aba6-799c6b8c945f-hosts-file\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14481b43-9d55-419e-9072-e2bb2e0f54d8-agent-certs\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008897 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/088c7d44-3d7c-4573-8850-8af99b26bb54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008907 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-node-log\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008930 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-config\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008960 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dead7e1a-456c-4dc9-aba6-799c6b8c945f-hosts-file\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.008962 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-kubelet-config\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009069 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3ace0aec-617e-4dea-bc85-b2a201718832-kubelet-config\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.010149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009060 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.010586 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62ns\" (UniqueName: \"kubernetes.io/projected/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kube-api-access-b62ns\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.010586 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.009145 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:44.010586 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009181 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-binary-copy\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.010586 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.009315 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:44.509276334 +0000 UTC m=+2.987311760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:44.010586 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.009481 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/088c7d44-3d7c-4573-8850-8af99b26bb54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.013134 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.013116 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 09:26:44.016403 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.016377 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9fmc\" (UniqueName: \"kubernetes.io/projected/088c7d44-3d7c-4573-8850-8af99b26bb54-kube-api-access-f9fmc\") pod \"multus-additional-cni-plugins-gk9tm\" (UID: \"088c7d44-3d7c-4573-8850-8af99b26bb54\") " pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.016489 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.016383 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbjj6\" (UniqueName: \"kubernetes.io/projected/dead7e1a-456c-4dc9-aba6-799c6b8c945f-kube-api-access-kbjj6\") pod \"node-resolver-pw8nq\" (UID: \"dead7e1a-456c-4dc9-aba6-799c6b8c945f\") " pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.016900 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.016879 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzsb6\" (UniqueName: \"kubernetes.io/projected/6ed79517-5b5e-4b46-ab41-a0f2831fe646-kube-api-access-zzsb6\") pod \"node-ca-5tkgk\" (UID: \"6ed79517-5b5e-4b46-ab41-a0f2831fe646\") " pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.017380 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.017361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62ns\" (UniqueName: \"kubernetes.io/projected/6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf-kube-api-access-b62ns\") pod \"aws-ebs-csi-driver-node-cwd5n\" (UID: \"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.110343 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-os-release\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-multus\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-hostroot\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-etc-kubernetes\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17a11154-32e1-4161-b43f-26881d05f427-iptables-alerter-script\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110423 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-multus\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14481b43-9d55-419e-9072-e2bb2e0f54d8-agent-certs\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.110475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110453 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-hostroot\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110474 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-os-release\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-node-log\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110515 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-node-log\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110480 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-etc-kubernetes\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-config\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqbq\" (UniqueName: \"kubernetes.io/projected/328b120f-2c8a-4462-b843-47ad0e7c9d92-kube-api-access-8lqbq\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110616 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw9mh\" (UniqueName: \"kubernetes.io/projected/17a11154-32e1-4161-b43f-26881d05f427-kube-api-access-rw9mh\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110642 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110668 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.110806 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-kubernetes\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-sys\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110836 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-sys\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110714 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110808 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-kubernetes\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.110787 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.110959 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:44.610920394 +0000 UTC m=+3.088955819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.110869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-tuned\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111049 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17a11154-32e1-4161-b43f-26881d05f427-iptables-alerter-script\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111063 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-bin\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111130 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-lib-modules\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.111245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111158 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-var-lib-kubelet\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111182 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-systemd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111158 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-bin\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-var-lib-kubelet\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-netns\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111238 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-netns\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111264 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-lib-modules\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111271 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-systemd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111301 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-netns\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111274 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-run-netns\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-env-overrides\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-modprobe-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-etc-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-system-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2rnh\" (UniqueName: \"kubernetes.io/projected/bcd140d7-76f7-4094-a426-093973d47036-kube-api-access-b2rnh\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112021 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-etc-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-netd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111492 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-modprobe-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111536 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-system-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-cni-netd\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111544 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-k8s-cni-cncf-io\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111589 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-k8s-cni-cncf-io\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-cni-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111617 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-kubelet\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-ovn\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111712 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sslck\" (UniqueName: \"kubernetes.io/projected/bc0962d2-8ea1-4296-8e39-a4d59d12d652-kube-api-access-sslck\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111733 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-cni-binary-copy\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.112789 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111753 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-env-overrides\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-kubelet\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111711 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-run-ovn\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-socket-dir-parent\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-multus-daemon-config\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111813 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-kubelet\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17a11154-32e1-4161-b43f-26881d05f427-host-slash\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111880 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-kubelet\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111891 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17a11154-32e1-4161-b43f-26881d05f427-host-slash\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111937 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-socket-dir-parent\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysconfig\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111980 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-conf-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.111992 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysconfig\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-multus-certs\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112035 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-multus-conf-dir\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112043 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-systemd-units\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112081 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-run-multus-certs\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112098 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-systemd-units\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-script-lib\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-host\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-bin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-host\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112308 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-cni-binary-copy\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112289 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-host-var-lib-cni-bin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112334 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bcd140d7-76f7-4094-a426-093973d47036-multus-daemon-config\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14481b43-9d55-419e-9072-e2bb2e0f54d8-konnectivity-ca\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112377 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-config\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-slash\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-host-slash\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112474 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.113934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-systemd\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-run\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-cnibin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112583 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-conf\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112609 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-tmp\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112620 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-d\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-log-socket\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovn-node-metrics-cert\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112668 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovnkube-script-lib\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tf6p\" (UniqueName: \"kubernetes.io/projected/548dabe2-0b7f-42a2-9f36-833d0943b948-kube-api-access-5tf6p\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-var-lib-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112732 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-systemd\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112692 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcd140d7-76f7-4094-a426-093973d47036-cnibin\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112884 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-sysctl-conf\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.112884 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/14481b43-9d55-419e-9072-e2bb2e0f54d8-konnectivity-ca\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.114396 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.113053 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-var-lib-openvswitch\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.113088 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/548dabe2-0b7f-42a2-9f36-833d0943b948-run\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.113093 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/328b120f-2c8a-4462-b843-47ad0e7c9d92-log-socket\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.114858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.113297 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-etc-tuned\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.114858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.113510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/14481b43-9d55-419e-9072-e2bb2e0f54d8-agent-certs\") pod \"konnectivity-agent-pwxws\" (UID: \"14481b43-9d55-419e-9072-e2bb2e0f54d8\") " pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.114858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.114678 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/548dabe2-0b7f-42a2-9f36-833d0943b948-tmp\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.115089 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.115006 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/328b120f-2c8a-4462-b843-47ad0e7c9d92-ovn-node-metrics-cert\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.117707 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.117686 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:44.117707 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.117707 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:44.117850 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.117717 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:44.117850 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.117782 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:44.617769715 +0000 UTC m=+3.095805117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:44.119837 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.119803 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw9mh\" (UniqueName: \"kubernetes.io/projected/17a11154-32e1-4161-b43f-26881d05f427-kube-api-access-rw9mh\") pod \"iptables-alerter-vnvfh\" (UID: \"17a11154-32e1-4161-b43f-26881d05f427\") " pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.120098 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.120078 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqbq\" (UniqueName: \"kubernetes.io/projected/328b120f-2c8a-4462-b843-47ad0e7c9d92-kube-api-access-8lqbq\") pod \"ovnkube-node-v7dl8\" (UID: \"328b120f-2c8a-4462-b843-47ad0e7c9d92\") " pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.120676 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.120652 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2rnh\" (UniqueName: \"kubernetes.io/projected/bcd140d7-76f7-4094-a426-093973d47036-kube-api-access-b2rnh\") pod \"multus-76qwn\" (UID: \"bcd140d7-76f7-4094-a426-093973d47036\") " pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.121115 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.121095 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tf6p\" (UniqueName: \"kubernetes.io/projected/548dabe2-0b7f-42a2-9f36-833d0943b948-kube-api-access-5tf6p\") pod \"tuned-h46bl\" (UID: \"548dabe2-0b7f-42a2-9f36-833d0943b948\") " pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.121884 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.121868 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sslck\" (UniqueName: \"kubernetes.io/projected/bc0962d2-8ea1-4296-8e39-a4d59d12d652-kube-api-access-sslck\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.211644 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.211584 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" Jul 14 09:26:44.218597 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.218575 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tkgk" Jul 14 09:26:44.220127 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.220105 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod088c7d44_3d7c_4573_8850_8af99b26bb54.slice/crio-2034fb5f68e06cd3caeec84c295b02a470b2d8849da8ca4e9833e5a2fd77c189 WatchSource:0}: Error finding container 2034fb5f68e06cd3caeec84c295b02a470b2d8849da8ca4e9833e5a2fd77c189: Status 404 returned error can't find the container with id 2034fb5f68e06cd3caeec84c295b02a470b2d8849da8ca4e9833e5a2fd77c189 Jul 14 09:26:44.226307 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.226284 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed79517_5b5e_4b46_ab41_a0f2831fe646.slice/crio-b83b2a2c7b29a863df91e398f83430e23c1319b24fefa2046c4fba7c35223452 WatchSource:0}: Error finding container b83b2a2c7b29a863df91e398f83430e23c1319b24fefa2046c4fba7c35223452: Status 404 returned error can't find the container with id b83b2a2c7b29a863df91e398f83430e23c1319b24fefa2046c4fba7c35223452 Jul 14 09:26:44.227373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.227354 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" Jul 14 09:26:44.231385 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.231361 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw8nq" Jul 14 09:26:44.235974 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.235943 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd4bea3_c7d5_4e8d_bd6f_893a4704d5cf.slice/crio-4bf920ce5db6c058f2cdb2e2e6ba6f8bc4c2bb88b8fe4135eafee0c3c17714f9 WatchSource:0}: Error finding container 4bf920ce5db6c058f2cdb2e2e6ba6f8bc4c2bb88b8fe4135eafee0c3c17714f9: Status 404 returned error can't find the container with id 4bf920ce5db6c058f2cdb2e2e6ba6f8bc4c2bb88b8fe4135eafee0c3c17714f9 Jul 14 09:26:44.237473 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.237448 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-76qwn" Jul 14 09:26:44.240465 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.240443 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddead7e1a_456c_4dc9_aba6_799c6b8c945f.slice/crio-cf8d2ee3143a20c2e3c52757dc309cadbff422d5e35aa8ec650af47f1b649c2b WatchSource:0}: Error finding container cf8d2ee3143a20c2e3c52757dc309cadbff422d5e35aa8ec650af47f1b649c2b: Status 404 returned error can't find the container with id cf8d2ee3143a20c2e3c52757dc309cadbff422d5e35aa8ec650af47f1b649c2b Jul 14 09:26:44.242378 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.242361 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:26:44.246032 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.246010 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd140d7_76f7_4094_a426_093973d47036.slice/crio-9528ddbaf4824501a153c7d513eac58cfe2c073839b1555666048c9b51c624e4 WatchSource:0}: Error finding container 9528ddbaf4824501a153c7d513eac58cfe2c073839b1555666048c9b51c624e4: Status 404 returned error can't find the container with id 9528ddbaf4824501a153c7d513eac58cfe2c073839b1555666048c9b51c624e4 Jul 14 09:26:44.247938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.247921 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:26:44.251155 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.250941 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328b120f_2c8a_4462_b843_47ad0e7c9d92.slice/crio-c819dcb4602dc2d50f252c55ae86d1a8ce0edb09d3ba7dddc4c89136c2ca0cdb WatchSource:0}: Error finding container c819dcb4602dc2d50f252c55ae86d1a8ce0edb09d3ba7dddc4c89136c2ca0cdb: Status 404 returned error can't find the container with id c819dcb4602dc2d50f252c55ae86d1a8ce0edb09d3ba7dddc4c89136c2ca0cdb Jul 14 09:26:44.254360 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.254335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vnvfh" Jul 14 09:26:44.255914 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.255895 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14481b43_9d55_419e_9072_e2bb2e0f54d8.slice/crio-b3d1a1fdf92f07638d83a7680ad71cb3960348191832c13954f087f982f3c325 WatchSource:0}: Error finding container b3d1a1fdf92f07638d83a7680ad71cb3960348191832c13954f087f982f3c325: Status 404 returned error can't find the container with id b3d1a1fdf92f07638d83a7680ad71cb3960348191832c13954f087f982f3c325 Jul 14 09:26:44.259902 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.259648 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h46bl" Jul 14 09:26:44.263396 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.263376 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a11154_32e1_4161_b43f_26881d05f427.slice/crio-0739df9cbf981da22b3f857f884cacae4265301d0f46323592dca303aa9e9431 WatchSource:0}: Error finding container 0739df9cbf981da22b3f857f884cacae4265301d0f46323592dca303aa9e9431: Status 404 returned error can't find the container with id 0739df9cbf981da22b3f857f884cacae4265301d0f46323592dca303aa9e9431 Jul 14 09:26:44.268235 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:26:44.268215 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548dabe2_0b7f_42a2_9f36_833d0943b948.slice/crio-710081dc4d4beb85441c3f060a5221e4a2e1065f674deec3479355f771acc0ca WatchSource:0}: Error finding container 710081dc4d4beb85441c3f060a5221e4a2e1065f674deec3479355f771acc0ca: Status 404 returned error can't find the container with id 710081dc4d4beb85441c3f060a5221e4a2e1065f674deec3479355f771acc0ca Jul 14 09:26:44.442480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.442372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" event={"ID":"f7d00eeede512e39bcca5885b1a3feaa","Type":"ContainerStarted","Data":"22ccd2740ca104c5d4a793d3de015691d57e29e2f4b6a2029b14db98cf7e03a5"} Jul 14 09:26:44.443796 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.443762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" event={"ID":"44c9368520183df5fbab72347490a409","Type":"ContainerStarted","Data":"a04662e4b305f9c355cfc78d8058d97c5432050a1dfd2928bc37a8d00635a581"} Jul 14 09:26:44.445579 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.445551 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h46bl" event={"ID":"548dabe2-0b7f-42a2-9f36-833d0943b948","Type":"ContainerStarted","Data":"710081dc4d4beb85441c3f060a5221e4a2e1065f674deec3479355f771acc0ca"} Jul 14 09:26:44.447064 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.447033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vnvfh" event={"ID":"17a11154-32e1-4161-b43f-26881d05f427","Type":"ContainerStarted","Data":"0739df9cbf981da22b3f857f884cacae4265301d0f46323592dca303aa9e9431"} Jul 14 09:26:44.448529 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.448480 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pwxws" event={"ID":"14481b43-9d55-419e-9072-e2bb2e0f54d8","Type":"ContainerStarted","Data":"b3d1a1fdf92f07638d83a7680ad71cb3960348191832c13954f087f982f3c325"} Jul 14 09:26:44.450047 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.450019 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"c819dcb4602dc2d50f252c55ae86d1a8ce0edb09d3ba7dddc4c89136c2ca0cdb"} Jul 14 09:26:44.451541 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.451498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-76qwn" event={"ID":"bcd140d7-76f7-4094-a426-093973d47036","Type":"ContainerStarted","Data":"9528ddbaf4824501a153c7d513eac58cfe2c073839b1555666048c9b51c624e4"} Jul 14 09:26:44.453375 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.453346 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw8nq" event={"ID":"dead7e1a-456c-4dc9-aba6-799c6b8c945f","Type":"ContainerStarted","Data":"cf8d2ee3143a20c2e3c52757dc309cadbff422d5e35aa8ec650af47f1b649c2b"} Jul 14 09:26:44.454948 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.454920 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" event={"ID":"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf","Type":"ContainerStarted","Data":"4bf920ce5db6c058f2cdb2e2e6ba6f8bc4c2bb88b8fe4135eafee0c3c17714f9"} Jul 14 09:26:44.456339 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.456313 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerStarted","Data":"2034fb5f68e06cd3caeec84c295b02a470b2d8849da8ca4e9833e5a2fd77c189"} Jul 14 09:26:44.457708 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.457680 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tkgk" event={"ID":"6ed79517-5b5e-4b46-ab41-a0f2831fe646","Type":"ContainerStarted","Data":"b83b2a2c7b29a863df91e398f83430e23c1319b24fefa2046c4fba7c35223452"} Jul 14 09:26:44.517060 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.516982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:44.517215 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.517198 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:44.517303 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.517284 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:45.517263973 +0000 UTC m=+3.995299393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:44.617582 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.617549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:44.618410 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.617740 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:44.618410 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.617811 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:45.617789987 +0000 UTC m=+4.095825394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:44.718673 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.718636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:44.718834 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.718823 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:44.718928 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.718842 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:44.718989 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.718855 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:44.719190 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:44.719168 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:45.719081742 +0000 UTC m=+4.197117169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:44.949078 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.948940 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:21:44 +0000 UTC" deadline="2028-01-05 22:56:04.018096027 +0000 UTC" Jul 14 09:26:44.949078 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:44.949003 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12973h29m19.06909757s" Jul 14 09:26:45.437001 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.436891 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:45.437587 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.437047 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:45.437587 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.437502 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:45.437702 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.437654 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:45.437757 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.437732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:45.437831 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.437810 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:45.505516 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.505485 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 09:26:45.527627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.527588 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:45.527805 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.527783 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:45.527927 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.527914 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:47.527828937 +0000 UTC m=+6.005864356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:45.628995 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.628958 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:45.629212 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.629139 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:45.629212 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.629201 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:47.629181184 +0000 UTC m=+6.107216592 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:45.729928 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.729837 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:45.730092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.730043 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:45.730092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.730065 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:45.730092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.730077 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:45.730296 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:45.730137 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:47.730117066 +0000 UTC m=+6.208152491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:45.949359 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.949313 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:21:44 +0000 UTC" deadline="2028-02-16 13:22:23.192897331 +0000 UTC" Jul 14 09:26:45.949359 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:45.949353 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13971h55m37.243548731s" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.436848 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.436990 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.437062 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.437146 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.437194 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:47.437679 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.437302 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:47.546380 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.546338 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:47.546599 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.546560 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:47.546664 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.546629 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:51.54661075 +0000 UTC m=+10.024646160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:47.646907 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.646725 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:47.646907 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.646902 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:47.647170 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.646977 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:51.646954947 +0000 UTC m=+10.124990368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:47.748085 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:47.747999 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:47.748279 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.748245 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:47.748279 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.748266 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:47.748279 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.748279 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:47.748459 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:47.748345 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:51.748325261 +0000 UTC m=+10.226360668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:49.436267 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:49.436233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:49.436748 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:49.436374 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:49.437276 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:49.436858 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:49.437276 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:49.436968 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:49.437276 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:49.437043 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:49.437276 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:49.437135 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:51.436515 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.436474 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:51.436515 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.436519 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:51.437142 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.436471 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:51.437142 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.436639 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:51.437142 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.436746 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:51.437142 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.436835 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:51.582422 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.582353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:51.582595 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.582510 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:51.582595 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.582583 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:59.582559634 +0000 UTC m=+18.060595041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:51.683296 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.683125 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:51.683296 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.683302 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:51.683548 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.683365 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:59.683345395 +0000 UTC m=+18.161380811 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:51.785188 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:51.784551 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:51.785188 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.784732 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:51.785188 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.784753 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:51.785188 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.784767 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:51.785188 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:51.784822 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:26:59.784803315 +0000 UTC m=+18.262838737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:53.436927 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:53.436811 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:53.437610 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:53.436980 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:53.437610 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:53.437375 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:53.437610 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:53.437405 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:53.437610 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:53.437493 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:53.437610 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:53.437583 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:55.436544 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:55.436508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:55.437013 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:55.436508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:55.437013 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:55.436647 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:55.437013 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:55.436508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:55.437013 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:55.436712 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:55.437013 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:55.436772 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:57.437106 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:57.437075 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:57.437538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:57.437116 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:57.437538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:57.437083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:57.437538 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:57.437220 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:57.437538 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:57.437359 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:57.437538 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:57.437440 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:59.436744 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.436711 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:59.437199 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.436753 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:59.437199 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.436755 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:59.437199 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.436831 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:26:59.437199 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.436946 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:26:59.437199 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.437026 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:26:59.650051 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.650017 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:26:59.650227 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.650168 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:59.650288 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.650253 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:15.650234871 +0000 UTC m=+34.128270279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:26:59.750977 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.750904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:26:59.751125 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.751061 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:59.751125 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.751123 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:15.751106836 +0000 UTC m=+34.229142238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:26:59.851577 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:26:59.851539 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:26:59.851735 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.851693 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:26:59.851735 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.851722 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:26:59.851735 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.851735 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:26:59.851861 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:26:59.851800 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:15.851778674 +0000 UTC m=+34.329814094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:27:01.437117 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:01.437084 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:01.437568 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:01.437096 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:01.437568 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:01.437211 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:01.437568 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:01.437315 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:01.437568 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:01.437102 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:01.437568 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:01.437418 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:03.437119 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:03.437081 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:03.437561 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:03.437222 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:03.437561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:03.437274 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:03.437561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:03.437290 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:03.437561 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:03.437384 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:03.437561 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:03.437469 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:05.437139 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.436966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:05.437629 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.437006 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:05.437629 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:05.437243 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:05.437629 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.437008 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:05.437629 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:05.437334 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:05.437629 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:05.437365 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:05.508224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.508177 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"04f42ee45825649d4d15c83e913bda3c3de8384eb24da831a62d47d94fbd1ddb"} Jul 14 09:27:05.510680 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.510648 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-76qwn" event={"ID":"bcd140d7-76f7-4094-a426-093973d47036","Type":"ContainerStarted","Data":"b57973f32eac4b95fc164035896a91ca6b46e19086997aa6090a53373ec34b9d"} Jul 14 09:27:05.512231 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.512202 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" event={"ID":"44c9368520183df5fbab72347490a409","Type":"ContainerStarted","Data":"e051c171f272b2b0ba6f3c53f968b2e076c5f51eb58cb19c1cc4f75e679ec2d8"} Jul 14 09:27:05.513619 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.513599 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h46bl" event={"ID":"548dabe2-0b7f-42a2-9f36-833d0943b948","Type":"ContainerStarted","Data":"ec031b5e91f07b03ca1f11fc0b03497bb5c42a96c536c1a61ef1de92474fa460"} Jul 14 09:27:05.529197 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.529147 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-76qwn" podStartSLOduration=1.821072886 podStartE2EDuration="22.52912742s" totalImagesPullingTime="20.708054534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:05.52880017 +0000 UTC m=+24.006835596" watchObservedRunningTime="2026-07-14 09:27:05.52912742 +0000 UTC m=+24.007162864" Jul 14 09:27:05.549663 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.549625 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h46bl" podStartSLOduration=1.895358461 podStartE2EDuration="22.549611074s" totalImagesPullingTime="20.654252613s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:05.549389516 +0000 UTC m=+24.027424942" watchObservedRunningTime="2026-07-14 09:27:05.549611074 +0000 UTC m=+24.027646498" Jul 14 09:27:05.563961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:05.563909 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-15.ec2.internal" podStartSLOduration=21.563889972 podStartE2EDuration="21.563889972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:05.563636497 +0000 UTC m=+24.041671920" watchObservedRunningTime="2026-07-14 09:27:05.563889972 +0000 UTC m=+24.041925397" Jul 14 09:27:06.516820 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.516594 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw8nq" event={"ID":"dead7e1a-456c-4dc9-aba6-799c6b8c945f","Type":"ContainerStarted","Data":"a590e4596b274bc69472aa0389ca66e8d708a8e3204c296f79c1c711d1ec6ccb"} Jul 14 09:27:06.518088 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.518056 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" event={"ID":"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf","Type":"ContainerStarted","Data":"122e982d5b6abdd9fbcf2c84d2b159fdfd5089fc85dda1aeaa30302c6e382ed9"} Jul 14 09:27:06.519512 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.519488 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="d6557d4f564eae0c97135fec7f69b0f95b3cf9ae0793a9c6d6185fae6440487f" exitCode=0 Jul 14 09:27:06.519584 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.519569 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"d6557d4f564eae0c97135fec7f69b0f95b3cf9ae0793a9c6d6185fae6440487f"} Jul 14 09:27:06.521099 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.521072 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tkgk" event={"ID":"6ed79517-5b5e-4b46-ab41-a0f2831fe646","Type":"ContainerStarted","Data":"8c13e94818ffe3cb2a851ab158b4e6cf68fb822a1eacd7888d599a192b3a7825"} Jul 14 09:27:06.522654 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.522630 2576 generic.go:356] "Generic (PLEG): container finished" podID="f7d00eeede512e39bcca5885b1a3feaa" containerID="5c0d98b71ad029936e5015dc7e5e9bebb45e48d1f109bb7d4e224c6663074554" exitCode=0 Jul 14 09:27:06.522738 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.522671 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" event={"ID":"f7d00eeede512e39bcca5885b1a3feaa","Type":"ContainerDied","Data":"5c0d98b71ad029936e5015dc7e5e9bebb45e48d1f109bb7d4e224c6663074554"} Jul 14 09:27:06.524152 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.524133 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vnvfh" event={"ID":"17a11154-32e1-4161-b43f-26881d05f427","Type":"ContainerStarted","Data":"9413049a805663c93856c9690a9c71aeb6f909a572123bced05eb01f0f98b363"} Jul 14 09:27:06.525538 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.525515 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pwxws" event={"ID":"14481b43-9d55-419e-9072-e2bb2e0f54d8","Type":"ContainerStarted","Data":"9878468aa393ddb63135b6f3cff14829956ba581d0815c9ca7d4e22f37e75ee8"} Jul 14 09:27:06.527982 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.527967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:27:06.528264 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528249 2576 generic.go:356] "Generic (PLEG): container finished" podID="328b120f-2c8a-4462-b843-47ad0e7c9d92" containerID="8d036881008d0618380fc43b60a4840c1aff70964a85e653b92729430fc477bb" exitCode=1 Jul 14 09:27:06.528341 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528324 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"0f22706cbe3ada5fa88f6c187b164e667a402dfd31f8871b9c7955927eed4708"} Jul 14 09:27:06.528386 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528377 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"9dc5396fc6ff3be6cc5a05296c03723d168182a4f1882342990c46b25c0b146f"} Jul 14 09:27:06.528429 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528407 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"b519bc1e3809cc6e4ee1ca70e2a4504e20ae13b945ce482b1189554a8c92f37f"} Jul 14 09:27:06.528461 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528435 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"0080db91c5feb9568f5d24a90331b9fbcba76655b75f79037a0fe2f6ff7b61cd"} Jul 14 09:27:06.528495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.528462 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerDied","Data":"8d036881008d0618380fc43b60a4840c1aff70964a85e653b92729430fc477bb"} Jul 14 09:27:06.533249 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.533214 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pw8nq" podStartSLOduration=2.853932092 podStartE2EDuration="23.533202658s" totalImagesPullingTime="20.679270566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:06.53241243 +0000 UTC m=+25.010447854" watchObservedRunningTime="2026-07-14 09:27:06.533202658 +0000 UTC m=+25.011238075" Jul 14 09:27:06.560944 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.560904 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pwxws" podStartSLOduration=2.914458498 podStartE2EDuration="23.560891251s" totalImagesPullingTime="20.646432753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:06.546310768 +0000 UTC m=+25.024346193" watchObservedRunningTime="2026-07-14 09:27:06.560891251 +0000 UTC m=+25.038926675" Jul 14 09:27:06.600640 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.600555 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vnvfh" podStartSLOduration=2.940358044 podStartE2EDuration="23.60053869s" totalImagesPullingTime="20.660180646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:06.600239522 +0000 UTC m=+25.078274946" watchObservedRunningTime="2026-07-14 09:27:06.60053869 +0000 UTC m=+25.078574114" Jul 14 09:27:06.616015 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:06.615966 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5tkgk" podStartSLOduration=2.938956541 podStartE2EDuration="23.615949138s" totalImagesPullingTime="20.676992597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:06.615645806 +0000 UTC m=+25.093681236" watchObservedRunningTime="2026-07-14 09:27:06.615949138 +0000 UTC m=+25.093984564" Jul 14 09:27:07.141376 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.141124 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 09:27:07.436764 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.436730 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:07.437092 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.436772 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:07.437092 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.436846 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:07.437092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:07.436860 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:07.437092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:07.436919 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:07.437092 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:07.437036 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:07.532788 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.532750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" event={"ID":"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf","Type":"ContainerStarted","Data":"8213ff4e7b19a77b6ac6f148f521d0f9288c3ee0f35796d930912ea3593273ac"} Jul 14 09:27:07.534823 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.534781 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" event={"ID":"f7d00eeede512e39bcca5885b1a3feaa","Type":"ContainerStarted","Data":"b8b94629c544fcdd54a6a89a7e1ae8125370381fd42a4bad9abe34e4a2639d98"} Jul 14 09:27:07.551454 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.551401 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-15.ec2.internal" podStartSLOduration=23.551386361 podStartE2EDuration="23.551386361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:07.550821958 +0000 UTC m=+26.028857383" watchObservedRunningTime="2026-07-14 09:27:07.551386361 +0000 UTC m=+26.029421786" Jul 14 09:27:07.990285 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.990151 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T09:27:07.141291615Z","UUID":"5110f52f-3e84-4b6e-97f4-37a1580bd4b6","Handler":null,"Name":"","Endpoint":""} Jul 14 09:27:07.992036 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.991911 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 09:27:07.992036 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:07.991954 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 09:27:08.541090 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:08.541069 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:27:08.541576 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:08.541542 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"1a635d915b32d7fa39ffb54f7ffbf4a76fe790a4d4d0c0975ece9489e708195b"} Jul 14 09:27:08.910039 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:08.909964 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:27:08.910580 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:08.910555 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:27:09.436625 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.436594 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:09.436794 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.436628 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:09.436794 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.436629 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:09.436794 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:09.436718 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:09.436920 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:09.436804 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:09.436920 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:09.436858 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:09.546053 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.546005 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" event={"ID":"6dd4bea3-c7d5-4e8d-bd6f-893a4704d5cf","Type":"ContainerStarted","Data":"2deb6383457f069f2d124be1555edf2cd4e8beb6fa49eede188cc026494aaa6b"} Jul 14 09:27:09.546627 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.546175 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:27:09.546706 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.546684 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pwxws" Jul 14 09:27:09.567685 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:09.567611 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cwd5n" podStartSLOduration=3.424326983 podStartE2EDuration="26.567594593s" totalImagesPullingTime="23.14326761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:09.567297919 +0000 UTC m=+28.045333344" watchObservedRunningTime="2026-07-14 09:27:09.567594593 +0000 UTC m=+28.045630018" Jul 14 09:27:11.436637 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.436613 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:11.436637 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.436623 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:11.437177 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.436623 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:11.437177 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:11.436703 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:11.437177 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:11.436779 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:11.437177 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:11.436838 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:11.553259 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.553238 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:27:11.553626 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.553609 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"fb379fb995ad166b2eb2a2b24de9b99029916095d723b73096b4c5c6fa1f259d"} Jul 14 09:27:11.553882 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.553851 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:11.554103 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.553923 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:11.554103 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.553996 2576 scope.go:117] "RemoveContainer" containerID="8d036881008d0618380fc43b60a4840c1aff70964a85e653b92729430fc477bb" Jul 14 09:27:11.555687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.555663 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="ac82763ac9327050a181f5dc43eeace557b7ba2da2d3eb3e111ee174687d2579" exitCode=0 Jul 14 09:27:11.555772 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.555740 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"ac82763ac9327050a181f5dc43eeace557b7ba2da2d3eb3e111ee174687d2579"} Jul 14 09:27:11.572131 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:11.572108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:12.565900 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.565740 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:27:12.566346 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.566275 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" event={"ID":"328b120f-2c8a-4462-b843-47ad0e7c9d92","Type":"ContainerStarted","Data":"c56cce3eec4d7bde8544489bfc98aa2972a3f903a9864ca79a4cbb698e33b270"} Jul 14 09:27:12.566587 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.566572 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:12.580922 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.580897 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:12.604780 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.604728 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" podStartSLOduration=8.505499827 podStartE2EDuration="29.604713545s" totalImagesPullingTime="21.099213718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:12.603282111 +0000 UTC m=+31.081317536" watchObservedRunningTime="2026-07-14 09:27:12.604713545 +0000 UTC m=+31.082748968" Jul 14 09:27:12.852984 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.852903 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s2jnc"] Jul 14 09:27:12.853092 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.853044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:12.853233 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:12.853173 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:12.853571 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.853556 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kcts2"] Jul 14 09:27:12.853645 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.853630 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:12.853705 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:12.853689 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:12.864907 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.864886 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qh6lf"] Jul 14 09:27:12.865007 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:12.864973 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:12.865065 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:12.865047 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:13.570620 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:13.570589 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="2fb4c9c7ab55ed6535e42338f064144a10b36b70ca44ccac87bb5ad2005b19a1" exitCode=0 Jul 14 09:27:13.571154 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:13.570667 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"2fb4c9c7ab55ed6535e42338f064144a10b36b70ca44ccac87bb5ad2005b19a1"} Jul 14 09:27:14.436239 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:14.436215 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:14.436338 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:14.436258 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:14.436338 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:14.436215 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:14.436441 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:14.436363 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:14.436441 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:14.436412 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:14.436542 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:14.436489 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:15.577029 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:15.576997 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="ab4a039887528e4416687c6a9ed8109b18460407c474c6b0cc0e0d8563fa3042" exitCode=0 Jul 14 09:27:15.577029 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:15.577032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"ab4a039887528e4416687c6a9ed8109b18460407c474c6b0cc0e0d8563fa3042"} Jul 14 09:27:15.678053 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:15.678015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:15.678173 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.678130 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 09:27:15.678244 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.678198 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret podName:3ace0aec-617e-4dea-bc85-b2a201718832 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:47.678165669 +0000 UTC m=+66.156201094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret") pod "global-pull-secret-syncer-s2jnc" (UID: "3ace0aec-617e-4dea-bc85-b2a201718832") : object "kube-system"/"original-pull-secret" not registered Jul 14 09:27:15.779427 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:15.779397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:15.779539 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.779494 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:27:15.779539 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.779536 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs podName:bc0962d2-8ea1-4296-8e39-a4d59d12d652 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:47.779523946 +0000 UTC m=+66.257559348 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs") pod "network-metrics-daemon-qh6lf" (UID: "bc0962d2-8ea1-4296-8e39-a4d59d12d652") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 09:27:15.879853 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:15.879784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:15.879972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.879889 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 09:27:15.879972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.879905 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 09:27:15.879972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.879914 2576 projected.go:196] Error preparing data for projected volume kube-api-access-ssg86 for pod openshift-network-diagnostics/network-check-target-kcts2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:27:15.879972 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:15.879951 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86 podName:9570dccb-8e40-4439-881d-53be44293564 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:47.879940466 +0000 UTC m=+66.357975869 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ssg86" (UniqueName: "kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86") pod "network-check-target-kcts2" (UID: "9570dccb-8e40-4439-881d-53be44293564") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 09:27:16.436221 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:16.436173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:16.436390 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:16.436173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:16.436390 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:16.436318 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kcts2" podUID="9570dccb-8e40-4439-881d-53be44293564" Jul 14 09:27:16.436390 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:16.436335 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:16.436541 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:16.436420 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qh6lf" podUID="bc0962d2-8ea1-4296-8e39-a4d59d12d652" Jul 14 09:27:16.436593 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:16.436542 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s2jnc" podUID="3ace0aec-617e-4dea-bc85-b2a201718832" Jul 14 09:27:17.800162 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.799959 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-15.ec2.internal" event="NodeReady" Jul 14 09:27:17.800523 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.800274 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 09:27:17.841922 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.841885 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:27:17.844017 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.843996 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.848194 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.848156 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 09:27:17.848419 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.848401 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 09:27:17.848728 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.848707 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-prphk\"" Jul 14 09:27:17.848824 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.848746 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 09:27:17.853978 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.853957 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 09:27:17.856845 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.856826 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-w558q"] Jul 14 09:27:17.858561 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.858544 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.859125 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.859108 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z8vwt"] Jul 14 09:27:17.860711 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.860691 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.862144 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.862117 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jncc8\"" Jul 14 09:27:17.862640 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.862609 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 09:27:17.862748 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.862649 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 09:27:17.863108 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.863090 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 09:27:17.863265 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.863210 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:27:17.863486 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.863471 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 09:27:17.863715 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.863702 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fxlg4\"" Jul 14 09:27:17.863840 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.863821 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 09:27:17.872730 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.872676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w558q"] Jul 14 09:27:17.874491 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.874466 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8vwt"] Jul 14 09:27:17.894106 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.894240 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894119 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894240 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894161 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894240 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrmpp\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894310 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad29e033-9aa7-4d33-8cb0-1eef8d354844-tmp-dir\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.894373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d39ef447-4a78-4e92-92a2-4f0252a4db58-cert\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9gwz\" (UniqueName: \"kubernetes.io/projected/d39ef447-4a78-4e92-92a2-4f0252a4db58-kube-api-access-s9gwz\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894444 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894466 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894491 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894512 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad29e033-9aa7-4d33-8cb0-1eef8d354844-config-volume\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.894533 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.894532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz8zn\" (UniqueName: \"kubernetes.io/projected/ad29e033-9aa7-4d33-8cb0-1eef8d354844-kube-api-access-qz8zn\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.994790 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994765 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.994938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994801 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrmpp\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.994938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad29e033-9aa7-4d33-8cb0-1eef8d354844-tmp-dir\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.994938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994845 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d39ef447-4a78-4e92-92a2-4f0252a4db58-cert\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.994938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9gwz\" (UniqueName: \"kubernetes.io/projected/d39ef447-4a78-4e92-92a2-4f0252a4db58-kube-api-access-s9gwz\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.994938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.994975 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:17.995013 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:17.995027 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56ccbcdf69-4bqhq: secret "image-registry-tls" not found Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995047 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad29e033-9aa7-4d33-8cb0-1eef8d354844-config-volume\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:17.995079 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls podName:a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:18.495061533 +0000 UTC m=+36.973096935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls") pod "image-registry-56ccbcdf69-4bqhq" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35") : secret "image-registry-tls" not found Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995098 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz8zn\" (UniqueName: \"kubernetes.io/projected/ad29e033-9aa7-4d33-8cb0-1eef8d354844-kube-api-access-qz8zn\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad29e033-9aa7-4d33-8cb0-1eef8d354844-tmp-dir\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.995215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995699 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995699 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995699 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:17.995348 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:17.995699 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:17.995403 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls podName:ad29e033-9aa7-4d33-8cb0-1eef8d354844 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:18.495378616 +0000 UTC m=+36.973414018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls") pod "dns-default-w558q" (UID: "ad29e033-9aa7-4d33-8cb0-1eef8d354844") : secret "dns-default-metrics-tls" not found Jul 14 09:27:17.995699 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995616 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad29e033-9aa7-4d33-8cb0-1eef8d354844-config-volume\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:17.995924 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995798 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.995924 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.995880 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.996347 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.996305 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.999512 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.999489 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:17.999512 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.999504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d39ef447-4a78-4e92-92a2-4f0252a4db58-cert\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:17.999721 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:17.999701 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:18.007716 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.007693 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9gwz\" (UniqueName: \"kubernetes.io/projected/d39ef447-4a78-4e92-92a2-4f0252a4db58-kube-api-access-s9gwz\") pod \"ingress-canary-z8vwt\" (UID: \"d39ef447-4a78-4e92-92a2-4f0252a4db58\") " pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:18.007996 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.007975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:18.009450 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.009430 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrmpp\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:18.009534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.009471 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz8zn\" (UniqueName: \"kubernetes.io/projected/ad29e033-9aa7-4d33-8cb0-1eef8d354844-kube-api-access-qz8zn\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:18.176874 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.176839 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8vwt" Jul 14 09:27:18.437047 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.437017 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:18.437252 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.437011 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:18.437252 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.437171 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:18.441438 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441413 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 09:27:18.441560 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441452 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 09:27:18.441560 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441474 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 09:27:18.441560 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441530 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frh48\"" Jul 14 09:27:18.441560 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441540 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ks9qb\"" Jul 14 09:27:18.441560 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.441474 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 09:27:18.500030 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.500001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:18.500158 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:18.500058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:18.500256 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:18.500163 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:18.500256 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:18.500197 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56ccbcdf69-4bqhq: secret "image-registry-tls" not found Jul 14 09:27:18.500256 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:18.500201 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:18.500256 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:18.500257 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls podName:a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:19.500237782 +0000 UTC m=+37.978273188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls") pod "image-registry-56ccbcdf69-4bqhq" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35") : secret "image-registry-tls" not found Jul 14 09:27:18.500436 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:18.500276 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls podName:ad29e033-9aa7-4d33-8cb0-1eef8d354844 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:19.500268312 +0000 UTC m=+37.978303715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls") pod "dns-default-w558q" (UID: "ad29e033-9aa7-4d33-8cb0-1eef8d354844") : secret "dns-default-metrics-tls" not found Jul 14 09:27:19.508908 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:19.508871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:19.509059 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:19.509080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:19.509121 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls podName:ad29e033-9aa7-4d33-8cb0-1eef8d354844 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.509106206 +0000 UTC m=+39.987141608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls") pod "dns-default-w558q" (UID: "ad29e033-9aa7-4d33-8cb0-1eef8d354844") : secret "dns-default-metrics-tls" not found Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:19.509228 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:19.509245 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56ccbcdf69-4bqhq: secret "image-registry-tls" not found Jul 14 09:27:19.509387 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:19.509313 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls podName:a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.509296512 +0000 UTC m=+39.987331918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls") pod "image-registry-56ccbcdf69-4bqhq" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35") : secret "image-registry-tls" not found Jul 14 09:27:19.702354 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:19.702327 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8vwt"] Jul 14 09:27:19.706102 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:19.706078 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39ef447_4a78_4e92_92a2_4f0252a4db58.slice/crio-6a4474de4478626d9cd4a244220e941639e6ffdf175440bde360b1c7e8a78be1 WatchSource:0}: Error finding container 6a4474de4478626d9cd4a244220e941639e6ffdf175440bde360b1c7e8a78be1: Status 404 returned error can't find the container with id 6a4474de4478626d9cd4a244220e941639e6ffdf175440bde360b1c7e8a78be1 Jul 14 09:27:20.049887 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.049806 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28"] Jul 14 09:27:20.051599 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.051578 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" Jul 14 09:27:20.054405 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.054384 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 09:27:20.055682 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.055665 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-cjqbr\"" Jul 14 09:27:20.055775 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.055694 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 09:27:20.064459 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.064441 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28"] Jul 14 09:27:20.114731 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.114703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql89h\" (UniqueName: \"kubernetes.io/projected/de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69-kube-api-access-ql89h\") pod \"migrator-7f59b54f6c-vjl28\" (UID: \"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" Jul 14 09:27:20.216091 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.216058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql89h\" (UniqueName: \"kubernetes.io/projected/de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69-kube-api-access-ql89h\") pod \"migrator-7f59b54f6c-vjl28\" (UID: \"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" Jul 14 09:27:20.241315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.241285 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql89h\" (UniqueName: \"kubernetes.io/projected/de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69-kube-api-access-ql89h\") pod \"migrator-7f59b54f6c-vjl28\" (UID: \"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" Jul 14 09:27:20.290176 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.290147 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mtqgg"] Jul 14 09:27:20.292285 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.292262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.295286 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.295253 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-runtime-extractor-mtqgg\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" podUID="ac2bb1f4-1f94-44e9-ae16-79ade319531a" pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.295403 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.295375 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 14 09:27:20.296230 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.296198 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-qsz5b\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsz5b\"" type="*v1.Secret" Jul 14 09:27:20.296745 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.296718 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 14 09:27:20.297077 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.297045 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" Jul 14 09:27:20.297166 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:20.297134 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" Jul 14 09:27:20.317102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.317039 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac2bb1f4-1f94-44e9-ae16-79ade319531a-crio-socket\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.317102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.317076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwds6\" (UniqueName: \"kubernetes.io/projected/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-api-access-pwds6\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.317266 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.317164 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac2bb1f4-1f94-44e9-ae16-79ade319531a-data-volume\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.317361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.317258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.317361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.317307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.318801 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.318779 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtqgg"] Jul 14 09:27:20.359894 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.359865 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" Jul 14 09:27:20.418215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.418123 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.418799 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.418769 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.418910 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.418877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac2bb1f4-1f94-44e9-ae16-79ade319531a-crio-socket\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.418949 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.418916 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwds6\" (UniqueName: \"kubernetes.io/projected/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-api-access-pwds6\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.419143 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.419011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac2bb1f4-1f94-44e9-ae16-79ade319531a-data-volume\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.419535 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.419358 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac2bb1f4-1f94-44e9-ae16-79ade319531a-crio-socket\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.419535 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.419419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac2bb1f4-1f94-44e9-ae16-79ade319531a-data-volume\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:20.492568 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.492534 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28"] Jul 14 09:27:20.495586 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:20.495546 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2bbe9c_4c6f_41e9_a6af_7398bd3e3b69.slice/crio-fed942d94283cd684ed9f8b28792bc79946f3a6c21f14895e79b175521cf83e4 WatchSource:0}: Error finding container fed942d94283cd684ed9f8b28792bc79946f3a6c21f14895e79b175521cf83e4: Status 404 returned error can't find the container with id fed942d94283cd684ed9f8b28792bc79946f3a6c21f14895e79b175521cf83e4 Jul 14 09:27:20.592174 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.591072 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" event={"ID":"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69","Type":"ContainerStarted","Data":"fed942d94283cd684ed9f8b28792bc79946f3a6c21f14895e79b175521cf83e4"} Jul 14 09:27:20.593047 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.592989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8vwt" event={"ID":"d39ef447-4a78-4e92-92a2-4f0252a4db58","Type":"ContainerStarted","Data":"6a4474de4478626d9cd4a244220e941639e6ffdf175440bde360b1c7e8a78be1"} Jul 14 09:27:20.599492 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.599457 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="ff31e94e84493945098b72c36f671f454c23923d90f3389bf34aad5f3b16a7b7" exitCode=0 Jul 14 09:27:20.599599 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:20.599497 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"ff31e94e84493945098b72c36f671f454c23923d90f3389bf34aad5f3b16a7b7"} Jul 14 09:27:21.195273 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.195247 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 09:27:21.378346 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.378315 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 09:27:21.378563 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.378536 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.378657 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.378620 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.878596829 +0000 UTC m=+40.356632251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.419003 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.418969 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: failed to sync configmap cache: timed out waiting for the condition Jul 14 09:27:21.419141 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.419065 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:21.919041104 +0000 UTC m=+40.397076511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : failed to sync configmap cache: timed out waiting for the condition Jul 14 09:27:21.526852 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.526774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:21.526852 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.526827 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:21.527060 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.526934 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:21.527060 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.526949 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56ccbcdf69-4bqhq: secret "image-registry-tls" not found Jul 14 09:27:21.527060 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.526957 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:21.527060 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.527000 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls podName:a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:25.526985997 +0000 UTC m=+44.005021400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls") pod "image-registry-56ccbcdf69-4bqhq" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35") : secret "image-registry-tls" not found Jul 14 09:27:21.527060 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.527014 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls podName:ad29e033-9aa7-4d33-8cb0-1eef8d354844 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:25.5270088 +0000 UTC m=+44.005044202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls") pod "dns-default-w558q" (UID: "ad29e033-9aa7-4d33-8cb0-1eef8d354844") : secret "dns-default-metrics-tls" not found Jul 14 09:27:21.606023 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.605994 2576 generic.go:356] "Generic (PLEG): container finished" podID="088c7d44-3d7c-4573-8850-8af99b26bb54" containerID="c4cb4a02ef956811eb1a06c8bb598b9ed88bbfa70ae8c2fc88b5fe5e7181a67a" exitCode=0 Jul 14 09:27:21.606650 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.606043 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerDied","Data":"c4cb4a02ef956811eb1a06c8bb598b9ed88bbfa70ae8c2fc88b5fe5e7181a67a"} Jul 14 09:27:21.834596 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.834521 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 09:27:21.836755 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.836736 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 09:27:21.849548 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.849526 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwds6\" (UniqueName: \"kubernetes.io/projected/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-api-access-pwds6\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:21.867985 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.867965 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qsz5b\"" Jul 14 09:27:21.930612 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.930581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:21.930754 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:21.930719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:21.930810 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.930755 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.930810 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.930801 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:21.930874 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.930831 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:22.930810446 +0000 UTC m=+41.408845855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:21.930874 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:21.930850 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:22.930839779 +0000 UTC m=+41.408875189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : configmap "kube-rbac-proxy" not found Jul 14 09:27:22.640326 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.640306 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjvp6"] Jul 14 09:27:22.660885 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.660867 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjvp6"] Jul 14 09:27:22.660955 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.660946 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.663625 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.663603 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 09:27:22.663726 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.663707 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 09:27:22.664705 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.664681 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 09:27:22.664705 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.664693 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-slzgm\"" Jul 14 09:27:22.664847 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.664779 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 09:27:22.680962 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.680938 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw8nq_dead7e1a-456c-4dc9-aba6-799c6b8c945f/dns-node-resolver/0.log" Jul 14 09:27:22.737994 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.737797 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hlhq\" (UniqueName: \"kubernetes.io/projected/9dd7dd94-dc62-47a2-90f4-aa443199ee38-kube-api-access-9hlhq\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.738103 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.738086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.738224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.738206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-key\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.838891 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.838866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hlhq\" (UniqueName: \"kubernetes.io/projected/9dd7dd94-dc62-47a2-90f4-aa443199ee38-kube-api-access-9hlhq\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.839018 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.838933 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.839018 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.838981 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-key\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.839659 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.839632 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-cabundle\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.842112 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.842084 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9dd7dd94-dc62-47a2-90f4-aa443199ee38-signing-key\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.847831 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.847811 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hlhq\" (UniqueName: \"kubernetes.io/projected/9dd7dd94-dc62-47a2-90f4-aa443199ee38-kube-api-access-9hlhq\") pod \"service-ca-79f68b9f4f-qjvp6\" (UID: \"9dd7dd94-dc62-47a2-90f4-aa443199ee38\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:22.940417 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.940388 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:22.940514 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.940429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:22.940514 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:22.940491 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:22.940575 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:22.940527 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:22.940575 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:22.940551 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:24.94053537 +0000 UTC m=+43.418570791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : configmap "kube-rbac-proxy" not found Jul 14 09:27:22.940575 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:22.940565 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:24.940559393 +0000 UTC m=+43.418594794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:22.970284 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:22.970262 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" Jul 14 09:27:23.129655 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.129626 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qjvp6"] Jul 14 09:27:23.132796 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:23.132586 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd7dd94_dc62_47a2_90f4_aa443199ee38.slice/crio-a94f05ca9ec572b66ee168c5c060928a5092fbe85b9cf884decd5b4a4471a4d6 WatchSource:0}: Error finding container a94f05ca9ec572b66ee168c5c060928a5092fbe85b9cf884decd5b4a4471a4d6: Status 404 returned error can't find the container with id a94f05ca9ec572b66ee168c5c060928a5092fbe85b9cf884decd5b4a4471a4d6 Jul 14 09:27:23.615873 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.615837 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" event={"ID":"088c7d44-3d7c-4573-8850-8af99b26bb54","Type":"ContainerStarted","Data":"76d2f5919c4e13d6d3492ff93e49767c9257a18f203f031cf61250a9d86b2c99"} Jul 14 09:27:23.617871 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.617836 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" event={"ID":"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69","Type":"ContainerStarted","Data":"0830e753d4ca1686186c6650aa7e093e8ec212e216f0a645f862c05f4be32ba6"} Jul 14 09:27:23.618001 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.617900 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" event={"ID":"de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69","Type":"ContainerStarted","Data":"7f391ef7534633ec93b1116bc8ea3f67a076fc8ac5ecd6eed1a861fe2a95d734"} Jul 14 09:27:23.619481 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.619458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8vwt" event={"ID":"d39ef447-4a78-4e92-92a2-4f0252a4db58","Type":"ContainerStarted","Data":"1d03c3e0d6a1cce867761973885f4f0d2bc620f3597e23fff4bd912adf013a27"} Jul 14 09:27:23.620823 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.620802 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" event={"ID":"9dd7dd94-dc62-47a2-90f4-aa443199ee38","Type":"ContainerStarted","Data":"a94f05ca9ec572b66ee168c5c060928a5092fbe85b9cf884decd5b4a4471a4d6"} Jul 14 09:27:23.644718 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.644671 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gk9tm" podStartSLOduration=9.235489996 podStartE2EDuration="40.64465658s" totalImagesPullingTime="31.409166584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:23.642978358 +0000 UTC m=+42.121013785" watchObservedRunningTime="2026-07-14 09:27:23.64465658 +0000 UTC m=+42.122691982" Jul 14 09:27:23.665746 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.665699 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z8vwt" podStartSLOduration=3.694229749 podStartE2EDuration="6.665683562s" totalImagesPullingTime="2.971453813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:23.665258768 +0000 UTC m=+42.143294193" watchObservedRunningTime="2026-07-14 09:27:23.665683562 +0000 UTC m=+42.143718989" Jul 14 09:27:23.682737 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.682688 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-vjl28" podStartSLOduration=1.4987918150000001 podStartE2EDuration="3.682675076s" totalImagesPullingTime="2.183883261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:23.682076579 +0000 UTC m=+42.160112003" watchObservedRunningTime="2026-07-14 09:27:23.682675076 +0000 UTC m=+42.160710499" Jul 14 09:27:23.684505 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:23.684482 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tkgk_6ed79517-5b5e-4b46-ab41-a0f2831fe646/node-ca/0.log" Jul 14 09:27:24.481744 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:24.481705 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8vwt_d39ef447-4a78-4e92-92a2-4f0252a4db58/serve-healthcheck-canary/0.log" Jul 14 09:27:24.682532 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:24.682503 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vjl28_de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69/migrator/0.log" Jul 14 09:27:24.881863 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:24.881789 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vjl28_de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69/graceful-termination/0.log" Jul 14 09:27:24.960306 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:24.960269 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:24.960465 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:24.960315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:24.960465 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:24.960420 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:24.960465 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:24.960449 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:24.960617 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:24.960499 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.960476832 +0000 UTC m=+47.438512240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : configmap "kube-rbac-proxy" not found Jul 14 09:27:24.960617 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:24.960519 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:28.960509091 +0000 UTC m=+47.438544499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:25.567056 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:25.567026 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:25.567174 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:25.567065 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:25.567174 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:25.567160 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 09:27:25.567174 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:25.567167 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 09:27:25.567291 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:25.567200 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56ccbcdf69-4bqhq: secret "image-registry-tls" not found Jul 14 09:27:25.567291 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:25.567221 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls podName:ad29e033-9aa7-4d33-8cb0-1eef8d354844 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:33.567209717 +0000 UTC m=+52.045245132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls") pod "dns-default-w558q" (UID: "ad29e033-9aa7-4d33-8cb0-1eef8d354844") : secret "dns-default-metrics-tls" not found Jul 14 09:27:25.567291 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:25.567241 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls podName:a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35 nodeName:}" failed. No retries permitted until 2026-07-14 09:27:33.567228 +0000 UTC m=+52.045263422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls") pod "image-registry-56ccbcdf69-4bqhq" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35") : secret "image-registry-tls" not found Jul 14 09:27:25.627476 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:25.627448 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" event={"ID":"9dd7dd94-dc62-47a2-90f4-aa443199ee38","Type":"ContainerStarted","Data":"1df2595b95e15013af224a8df0944222128e65cd7d79971382b269ecd035fd5f"} Jul 14 09:27:25.642907 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:25.642855 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-qjvp6" podStartSLOduration=1.284351425 podStartE2EDuration="3.642837524s" totalImagesPullingTime="2.358486099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:25.64267601 +0000 UTC m=+44.120711434" watchObservedRunningTime="2026-07-14 09:27:25.642837524 +0000 UTC m=+44.120872949" Jul 14 09:27:28.993102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:28.993073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:28.993102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:28.993108 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:28.993565 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:28.993232 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 09:27:28.993565 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:28.993257 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 09:27:28.993565 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:28.993304 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:36.993288335 +0000 UTC m=+55.471323751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : secret "insights-runtime-extractor-tls" not found Jul 14 09:27:28.993565 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:27:28.993317 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm podName:ac2bb1f4-1f94-44e9-ae16-79ade319531a nodeName:}" failed. No retries permitted until 2026-07-14 09:27:36.993310966 +0000 UTC m=+55.471346368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtqgg" (UID: "ac2bb1f4-1f94-44e9-ae16-79ade319531a") : configmap "kube-rbac-proxy" not found Jul 14 09:27:33.628823 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.628783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:33.629207 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.628845 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:33.631878 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.631848 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad29e033-9aa7-4d33-8cb0-1eef8d354844-metrics-tls\") pod \"dns-default-w558q\" (UID: \"ad29e033-9aa7-4d33-8cb0-1eef8d354844\") " pod="openshift-dns/dns-default-w558q" Jul 14 09:27:33.632138 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.632115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"image-registry-56ccbcdf69-4bqhq\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:33.754883 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.754855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:33.770231 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.770206 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-w558q" Jul 14 09:27:33.918795 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.918764 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:27:33.922504 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:33.922478 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c4e1a0_d9d4_46cf_837c_cd4fc13d3b35.slice/crio-173eed7f847dac8618449cc067708ef7acc6f2aafca7904acde7e3d05d39f57f WatchSource:0}: Error finding container 173eed7f847dac8618449cc067708ef7acc6f2aafca7904acde7e3d05d39f57f: Status 404 returned error can't find the container with id 173eed7f847dac8618449cc067708ef7acc6f2aafca7904acde7e3d05d39f57f Jul 14 09:27:33.938495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:33.938461 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-w558q"] Jul 14 09:27:33.941222 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:33.941176 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad29e033_9aa7_4d33_8cb0_1eef8d354844.slice/crio-21b8fb2f233d82f257b0013a550588c46c69b837939a4fa89351e1b659e27788 WatchSource:0}: Error finding container 21b8fb2f233d82f257b0013a550588c46c69b837939a4fa89351e1b659e27788: Status 404 returned error can't find the container with id 21b8fb2f233d82f257b0013a550588c46c69b837939a4fa89351e1b659e27788 Jul 14 09:27:34.653878 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:34.653842 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" event={"ID":"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35","Type":"ContainerStarted","Data":"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5"} Jul 14 09:27:34.654339 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:34.653907 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:27:34.654339 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:34.653929 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" event={"ID":"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35","Type":"ContainerStarted","Data":"173eed7f847dac8618449cc067708ef7acc6f2aafca7904acde7e3d05d39f57f"} Jul 14 09:27:34.655055 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:34.655028 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w558q" event={"ID":"ad29e033-9aa7-4d33-8cb0-1eef8d354844","Type":"ContainerStarted","Data":"21b8fb2f233d82f257b0013a550588c46c69b837939a4fa89351e1b659e27788"} Jul 14 09:27:34.673692 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:34.673646 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" podStartSLOduration=22.67362884 podStartE2EDuration="22.67362884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:34.673453512 +0000 UTC m=+53.151488937" watchObservedRunningTime="2026-07-14 09:27:34.67362884 +0000 UTC m=+53.151664263" Jul 14 09:27:36.662920 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:36.662845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w558q" event={"ID":"ad29e033-9aa7-4d33-8cb0-1eef8d354844","Type":"ContainerStarted","Data":"dc3fe3d19a735c65c4e99fd17b9570d9bf7adafb69b3795787a98884951687d3"} Jul 14 09:27:36.662920 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:36.662898 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-w558q" event={"ID":"ad29e033-9aa7-4d33-8cb0-1eef8d354844","Type":"ContainerStarted","Data":"c4d333f56e7e5177bda9aded75e6a2bdec3226408abe060cbb23abff62c5ba45"} Jul 14 09:27:36.663338 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:36.662984 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-w558q" Jul 14 09:27:36.681172 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:36.681127 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-w558q" podStartSLOduration=17.634946504 podStartE2EDuration="19.681114559s" totalImagesPullingTime="2.046168055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:36.680024697 +0000 UTC m=+55.158060145" watchObservedRunningTime="2026-07-14 09:27:36.681114559 +0000 UTC m=+55.159149983" Jul 14 09:27:37.056497 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.056470 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:37.056497 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.056502 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:37.056983 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.056955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac2bb1f4-1f94-44e9-ae16-79ade319531a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:37.058718 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.058693 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac2bb1f4-1f94-44e9-ae16-79ade319531a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtqgg\" (UID: \"ac2bb1f4-1f94-44e9-ae16-79ade319531a\") " pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:37.102773 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.102749 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtqgg" Jul 14 09:27:37.239377 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.239348 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtqgg"] Jul 14 09:27:37.243091 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:37.243064 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2bb1f4_1f94_44e9_ae16_79ade319531a.slice/crio-2059b239abb0c23ac1cb48bbbb53846f3474aecaae257e0c17c829750f6e8341 WatchSource:0}: Error finding container 2059b239abb0c23ac1cb48bbbb53846f3474aecaae257e0c17c829750f6e8341: Status 404 returned error can't find the container with id 2059b239abb0c23ac1cb48bbbb53846f3474aecaae257e0c17c829750f6e8341 Jul 14 09:27:37.666948 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:37.666916 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtqgg" event={"ID":"ac2bb1f4-1f94-44e9-ae16-79ade319531a","Type":"ContainerStarted","Data":"2059b239abb0c23ac1cb48bbbb53846f3474aecaae257e0c17c829750f6e8341"} Jul 14 09:27:41.682430 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:41.682401 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtqgg" event={"ID":"ac2bb1f4-1f94-44e9-ae16-79ade319531a","Type":"ContainerStarted","Data":"e7698c5a0de70a51664654e59db7c4da08094e5cd491864aa668998d0a434d27"} Jul 14 09:27:41.682781 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:41.682457 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtqgg" event={"ID":"ac2bb1f4-1f94-44e9-ae16-79ade319531a","Type":"ContainerStarted","Data":"705dbfb5c8a93f558232d7ca132e59f518de62b94a8ac42f9e32926fa9865bd6"} Jul 14 09:27:44.403452 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.402195 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds"] Jul 14 09:27:44.424241 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.424208 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66"] Jul 14 09:27:44.424390 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.424294 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.428007 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.427985 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 09:27:44.428136 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.428076 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 09:27:44.428382 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.428366 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 09:27:44.429255 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.429237 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 09:27:44.429332 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.429295 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ljjf9\"" Jul 14 09:27:44.439891 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.439865 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss"] Jul 14 09:27:44.439988 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.439960 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.442453 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.442434 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 09:27:44.456042 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.456023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.458723 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.458704 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 09:27:44.458822 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.458708 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 09:27:44.460087 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.459920 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 09:27:44.460087 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.459984 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 09:27:44.460610 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.460594 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66"] Jul 14 09:27:44.460656 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.460628 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds"] Jul 14 09:27:44.460656 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.460639 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss"] Jul 14 09:27:44.463765 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.463749 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:27:44.515813 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.515793 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/35db87f0-32b6-4780-991f-e06cb6f97418-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.515931 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.515831 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26svj\" (UniqueName: \"kubernetes.io/projected/89a5b758-a382-4d3c-8988-451ab1847056-kube-api-access-26svj\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.515931 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.515862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35db87f0-32b6-4780-991f-e06cb6f97418-tmp\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.515931 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.515910 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89a5b758-a382-4d3c-8988-451ab1847056-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.516048 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.515931 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bqk\" (UniqueName: \"kubernetes.io/projected/35db87f0-32b6-4780-991f-e06cb6f97418-kube-api-access-g5bqk\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.585790 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.585770 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v7dl8" Jul 14 09:27:44.594408 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.594387 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c647c656d-c22xh"] Jul 14 09:27:44.616285 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616266 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35db87f0-32b6-4780-991f-e06cb6f97418-tmp\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.616386 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89a5b758-a382-4d3c-8988-451ab1847056-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.616386 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616321 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bqk\" (UniqueName: \"kubernetes.io/projected/35db87f0-32b6-4780-991f-e06cb6f97418-kube-api-access-g5bqk\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.616386 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.616386 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616368 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db45d8bc-c1f9-430b-8839-b3ff45f171b1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.616563 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqhxp\" (UniqueName: \"kubernetes.io/projected/db45d8bc-c1f9-430b-8839-b3ff45f171b1-kube-api-access-nqhxp\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.616563 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616424 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.616563 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616539 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.616710 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616577 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/35db87f0-32b6-4780-991f-e06cb6f97418-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.616710 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616619 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/35db87f0-32b6-4780-991f-e06cb6f97418-tmp\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.616710 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26svj\" (UniqueName: \"kubernetes.io/projected/89a5b758-a382-4d3c-8988-451ab1847056-kube-api-access-26svj\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.616853 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.616752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.618765 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.618743 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/89a5b758-a382-4d3c-8988-451ab1847056-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.618974 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.618956 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/35db87f0-32b6-4780-991f-e06cb6f97418-klusterlet-config\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.619849 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.619831 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c647c656d-c22xh"] Jul 14 09:27:44.619923 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.619909 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.634565 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.634541 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bqk\" (UniqueName: \"kubernetes.io/projected/35db87f0-32b6-4780-991f-e06cb6f97418-kube-api-access-g5bqk\") pod \"klusterlet-addon-workmgr-55d597bc7-k8j66\" (UID: \"35db87f0-32b6-4780-991f-e06cb6f97418\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.634649 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.634543 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26svj\" (UniqueName: \"kubernetes.io/projected/89a5b758-a382-4d3c-8988-451ab1847056-kube-api-access-26svj\") pod \"managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds\" (UID: \"89a5b758-a382-4d3c-8988-451ab1847056\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.693562 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.693537 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtqgg" event={"ID":"ac2bb1f4-1f94-44e9-ae16-79ade319531a","Type":"ContainerStarted","Data":"de24c5fd765d34e314702a19327115b4fffec53b0e707b266c4e3873e95d6390"} Jul 14 09:27:44.714905 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.714868 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mtqgg" podStartSLOduration=17.911432582 podStartE2EDuration="24.714856833s" totalImagesPullingTime="6.803424251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:27:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:44.71446707 +0000 UTC m=+63.192502495" watchObservedRunningTime="2026-07-14 09:27:44.714856833 +0000 UTC m=+63.192892257" Jul 14 09:27:44.717835 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.717814 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-certificates\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.717948 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.717870 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.717948 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.717938 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.718057 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.717970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db45d8bc-c1f9-430b-8839-b3ff45f171b1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.718057 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.717988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhxp\" (UniqueName: \"kubernetes.io/projected/db45d8bc-c1f9-430b-8839-b3ff45f171b1-kube-api-access-nqhxp\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.718057 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718009 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.718057 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718043 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-installation-pull-secrets\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718292 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718069 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbjn\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-kube-api-access-rwbjn\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718292 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-image-registry-private-configuration\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718292 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718128 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-tls\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718292 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-bound-sa-token\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98ecac76-e91d-4f50-893c-7c4f891530ba-ca-trust-extracted\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718317 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-trusted-ca\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.718480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.718757 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.718734 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db45d8bc-c1f9-430b-8839-b3ff45f171b1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.720289 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.720265 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-ca\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.720384 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.720362 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.720652 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.720628 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.720743 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.720727 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db45d8bc-c1f9-430b-8839-b3ff45f171b1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.733987 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.733958 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhxp\" (UniqueName: \"kubernetes.io/projected/db45d8bc-c1f9-430b-8839-b3ff45f171b1-kube-api-access-nqhxp\") pod \"cluster-proxy-proxy-agent-55494496c5-fhhss\" (UID: \"db45d8bc-c1f9-430b-8839-b3ff45f171b1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.741024 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.741008 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" Jul 14 09:27:44.747373 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.747357 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:44.765281 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.765257 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" Jul 14 09:27:44.819669 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-trusted-ca\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.819809 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819728 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-certificates\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.819871 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-installation-pull-secrets\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.819871 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbjn\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-kube-api-access-rwbjn\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.819982 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-image-registry-private-configuration\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.819982 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-tls\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.820088 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.819985 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-bound-sa-token\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.820088 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.820023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98ecac76-e91d-4f50-893c-7c4f891530ba-ca-trust-extracted\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.820503 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.820476 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/98ecac76-e91d-4f50-893c-7c4f891530ba-ca-trust-extracted\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.821936 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.821361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-trusted-ca\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.823135 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.823090 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-certificates\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.825728 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.824884 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-registry-tls\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.827364 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.827342 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-installation-pull-secrets\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.827768 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.827721 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/98ecac76-e91d-4f50-893c-7c4f891530ba-image-registry-private-configuration\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.833155 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.833107 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-bound-sa-token\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.834887 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.834836 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbjn\" (UniqueName: \"kubernetes.io/projected/98ecac76-e91d-4f50-893c-7c4f891530ba-kube-api-access-rwbjn\") pod \"image-registry-7c647c656d-c22xh\" (UID: \"98ecac76-e91d-4f50-893c-7c4f891530ba\") " pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:44.904483 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.904421 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds"] Jul 14 09:27:44.907422 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:44.907390 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89a5b758_a382_4d3c_8988_451ab1847056.slice/crio-ab3454e242cc368195af109fdd0889c102906b14206664e7e080922aa820ac46 WatchSource:0}: Error finding container ab3454e242cc368195af109fdd0889c102906b14206664e7e080922aa820ac46: Status 404 returned error can't find the container with id ab3454e242cc368195af109fdd0889c102906b14206664e7e080922aa820ac46 Jul 14 09:27:44.915875 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.915849 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66"] Jul 14 09:27:44.918312 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:44.918288 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35db87f0_32b6_4780_991f_e06cb6f97418.slice/crio-bb5693ccc778f53759392009779b55451ff7deb439bff655c971911f1097e62e WatchSource:0}: Error finding container bb5693ccc778f53759392009779b55451ff7deb439bff655c971911f1097e62e: Status 404 returned error can't find the container with id bb5693ccc778f53759392009779b55451ff7deb439bff655c971911f1097e62e Jul 14 09:27:44.936320 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.936299 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss"] Jul 14 09:27:44.939015 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:44.938981 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb45d8bc_c1f9_430b_8839_b3ff45f171b1.slice/crio-a545ab102d66f8cdb54a900904baa830686f0eb6b3e76f2e8a5cca16662c512f WatchSource:0}: Error finding container a545ab102d66f8cdb54a900904baa830686f0eb6b3e76f2e8a5cca16662c512f: Status 404 returned error can't find the container with id a545ab102d66f8cdb54a900904baa830686f0eb6b3e76f2e8a5cca16662c512f Jul 14 09:27:44.957953 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:44.957908 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:45.099675 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.099645 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c647c656d-c22xh"] Jul 14 09:27:45.101755 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:45.101729 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ecac76_e91d_4f50_893c_7c4f891530ba.slice/crio-372bde70bc84802460f1ce927140336a67949834a8fafcd92d7b4b2f48e48705 WatchSource:0}: Error finding container 372bde70bc84802460f1ce927140336a67949834a8fafcd92d7b4b2f48e48705: Status 404 returned error can't find the container with id 372bde70bc84802460f1ce927140336a67949834a8fafcd92d7b4b2f48e48705 Jul 14 09:27:45.699667 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.699624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" event={"ID":"89a5b758-a382-4d3c-8988-451ab1847056","Type":"ContainerStarted","Data":"ab3454e242cc368195af109fdd0889c102906b14206664e7e080922aa820ac46"} Jul 14 09:27:45.701723 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.701695 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" event={"ID":"98ecac76-e91d-4f50-893c-7c4f891530ba","Type":"ContainerStarted","Data":"c22226860e585081fcf8f79e4a50506e808362aca81a7a39f221f7cc73d77508"} Jul 14 09:27:45.701860 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.701750 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" event={"ID":"98ecac76-e91d-4f50-893c-7c4f891530ba","Type":"ContainerStarted","Data":"372bde70bc84802460f1ce927140336a67949834a8fafcd92d7b4b2f48e48705"} Jul 14 09:27:45.702436 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.702389 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:27:45.703635 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.703614 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" event={"ID":"35db87f0-32b6-4780-991f-e06cb6f97418","Type":"ContainerStarted","Data":"bb5693ccc778f53759392009779b55451ff7deb439bff655c971911f1097e62e"} Jul 14 09:27:45.704724 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:45.704687 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" event={"ID":"db45d8bc-c1f9-430b-8839-b3ff45f171b1","Type":"ContainerStarted","Data":"a545ab102d66f8cdb54a900904baa830686f0eb6b3e76f2e8a5cca16662c512f"} Jul 14 09:27:46.669444 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:46.669393 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-w558q" Jul 14 09:27:46.688594 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:46.688536 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" podStartSLOduration=2.6885163739999998 podStartE2EDuration="2.688516374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:45.726601565 +0000 UTC m=+64.204636989" watchObservedRunningTime="2026-07-14 09:27:46.688516374 +0000 UTC m=+65.166551800" Jul 14 09:27:47.747358 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.747301 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:47.750169 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.750145 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 09:27:47.761734 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.761703 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3ace0aec-617e-4dea-bc85-b2a201718832-original-pull-secret\") pod \"global-pull-secret-syncer-s2jnc\" (UID: \"3ace0aec-617e-4dea-bc85-b2a201718832\") " pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:47.848978 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.848505 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:47.852028 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.852004 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 09:27:47.862662 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.862428 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s2jnc" Jul 14 09:27:47.863377 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.863352 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc0962d2-8ea1-4296-8e39-a4d59d12d652-metrics-certs\") pod \"network-metrics-daemon-qh6lf\" (UID: \"bc0962d2-8ea1-4296-8e39-a4d59d12d652\") " pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:47.952301 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.952205 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:47.955005 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.954977 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 09:27:47.965688 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.965527 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 09:27:47.981440 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:47.981385 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssg86\" (UniqueName: \"kubernetes.io/projected/9570dccb-8e40-4439-881d-53be44293564-kube-api-access-ssg86\") pod \"network-check-target-kcts2\" (UID: \"9570dccb-8e40-4439-881d-53be44293564\") " pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:48.152505 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.152480 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frh48\"" Jul 14 09:27:48.158253 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.158233 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ks9qb\"" Jul 14 09:27:48.160718 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.160677 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qh6lf" Jul 14 09:27:48.167721 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.166259 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:48.271584 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.271424 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s2jnc"] Jul 14 09:27:48.355872 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.355840 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qh6lf"] Jul 14 09:27:48.367700 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.367569 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kcts2"] Jul 14 09:27:48.722397 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.722358 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" event={"ID":"89a5b758-a382-4d3c-8988-451ab1847056","Type":"ContainerStarted","Data":"e55f6a2f8e40a0d33c99d1a22a6249500be714c55cf4eb16228e07b8717a9e24"} Jul 14 09:27:48.738960 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:48.738904 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d4bf8d6dd-nf2ds" podStartSLOduration=1.505492747 podStartE2EDuration="4.738889919s" totalImagesPullingTime="3.233397172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:48.737864074 +0000 UTC m=+67.215899499" watchObservedRunningTime="2026-07-14 09:27:48.738889919 +0000 UTC m=+67.216925342" Jul 14 09:27:49.442151 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:49.442119 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc0962d2_8ea1_4296_8e39_a4d59d12d652.slice/crio-692b75ae19ba1220307299bd96363ab838146c76a1ae44429e3266c839c496ff WatchSource:0}: Error finding container 692b75ae19ba1220307299bd96363ab838146c76a1ae44429e3266c839c496ff: Status 404 returned error can't find the container with id 692b75ae19ba1220307299bd96363ab838146c76a1ae44429e3266c839c496ff Jul 14 09:27:49.443250 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:49.443026 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ace0aec_617e_4dea_bc85_b2a201718832.slice/crio-bcfd18097302e5e64a3cec1a5a5dc16ffec55aede09d33db7975e751bc3896f8 WatchSource:0}: Error finding container bcfd18097302e5e64a3cec1a5a5dc16ffec55aede09d33db7975e751bc3896f8: Status 404 returned error can't find the container with id bcfd18097302e5e64a3cec1a5a5dc16ffec55aede09d33db7975e751bc3896f8 Jul 14 09:27:49.727829 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:49.727741 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s2jnc" event={"ID":"3ace0aec-617e-4dea-bc85-b2a201718832","Type":"ContainerStarted","Data":"bcfd18097302e5e64a3cec1a5a5dc16ffec55aede09d33db7975e751bc3896f8"} Jul 14 09:27:49.729018 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:49.728983 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qh6lf" event={"ID":"bc0962d2-8ea1-4296-8e39-a4d59d12d652","Type":"ContainerStarted","Data":"692b75ae19ba1220307299bd96363ab838146c76a1ae44429e3266c839c496ff"} Jul 14 09:27:50.558977 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:50.558946 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9570dccb_8e40_4439_881d_53be44293564.slice/crio-1bacd9da0c12dab4925b1642c1f9e3831c9db6262165bc66f101ae2eae661569 WatchSource:0}: Error finding container 1bacd9da0c12dab4925b1642c1f9e3831c9db6262165bc66f101ae2eae661569: Status 404 returned error can't find the container with id 1bacd9da0c12dab4925b1642c1f9e3831c9db6262165bc66f101ae2eae661569 Jul 14 09:27:50.741841 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.741617 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" event={"ID":"35db87f0-32b6-4780-991f-e06cb6f97418","Type":"ContainerStarted","Data":"b60a1c5b545c62b1e3481fd1932eedc5ffde441a425934730e4d21c4e9d54701"} Jul 14 09:27:50.741987 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.741877 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:50.744406 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.743956 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" Jul 14 09:27:50.744406 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.743994 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kcts2" event={"ID":"9570dccb-8e40-4439-881d-53be44293564","Type":"ContainerStarted","Data":"1bacd9da0c12dab4925b1642c1f9e3831c9db6262165bc66f101ae2eae661569"} Jul 14 09:27:50.748215 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.748078 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" event={"ID":"db45d8bc-c1f9-430b-8839-b3ff45f171b1","Type":"ContainerStarted","Data":"134b9938c08db582c09f064a8e560ef2993a8d6833a75d6e6a24c623d19620b9"} Jul 14 09:27:50.759435 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:50.759359 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55d597bc7-k8j66" podStartSLOduration=1.071035269 podStartE2EDuration="6.759338273s" totalImagesPullingTime="5.688303004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:50.75824983 +0000 UTC m=+69.236285246" watchObservedRunningTime="2026-07-14 09:27:50.759338273 +0000 UTC m=+69.237373691" Jul 14 09:27:51.754882 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:51.754845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qh6lf" event={"ID":"bc0962d2-8ea1-4296-8e39-a4d59d12d652","Type":"ContainerStarted","Data":"c0489527de53503518165d49f66b9f09c5ed36eb1ff554173d294c032b0e2b54"} Jul 14 09:27:51.761095 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:51.761038 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" event={"ID":"db45d8bc-c1f9-430b-8839-b3ff45f171b1","Type":"ContainerStarted","Data":"955034fd81bc5991ae7926f4fa5a0d2f0a07f7b2f07fafe502f3e87fc9054dee"} Jul 14 09:27:51.761231 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:51.761108 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" event={"ID":"db45d8bc-c1f9-430b-8839-b3ff45f171b1","Type":"ContainerStarted","Data":"eb845f3895d45cd6512129d703ae7dacd7bb73f72c91032ef18391e23030769a"} Jul 14 09:27:51.780421 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:51.780367 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-55494496c5-fhhss" podStartSLOduration=2.113617801 podStartE2EDuration="7.780346304s" totalImagesPullingTime="5.666728503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:51.779063141 +0000 UTC m=+70.257098567" watchObservedRunningTime="2026-07-14 09:27:51.780346304 +0000 UTC m=+70.258381732" Jul 14 09:27:52.766802 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:52.766758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qh6lf" event={"ID":"bc0962d2-8ea1-4296-8e39-a4d59d12d652","Type":"ContainerStarted","Data":"5ca1b17d73d9faee1cf6fb7cae1fda1c416dfb173d7ec4f1296969163ddddff1"} Jul 14 09:27:52.785090 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:52.785042 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qh6lf" podStartSLOduration=67.649262857 podStartE2EDuration="1m9.785025109s" totalImagesPullingTime="2.135762252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:52.783755495 +0000 UTC m=+71.261790920" watchObservedRunningTime="2026-07-14 09:27:52.785025109 +0000 UTC m=+71.263060535" Jul 14 09:27:54.469482 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:54.469443 2576 patch_prober.go:28] interesting pod/image-registry-56ccbcdf69-4bqhq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 09:27:54.469922 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:54.469504 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 09:27:55.779398 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:55.779360 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s2jnc" event={"ID":"3ace0aec-617e-4dea-bc85-b2a201718832","Type":"ContainerStarted","Data":"425a0d6f7da6c7e752f1dc3410b09d36bf9382bf4acdc12cf513ab836878a759"} Jul 14 09:27:55.780802 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:55.780781 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kcts2" event={"ID":"9570dccb-8e40-4439-881d-53be44293564","Type":"ContainerStarted","Data":"b2a54f3a8b318c8e221c92f586afd1b51406a0611098051b86c893f8c64f3c96"} Jul 14 09:27:55.780891 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:55.780860 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:27:55.799965 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:55.799916 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s2jnc" podStartSLOduration=67.086706412 podStartE2EDuration="1m12.799902235s" totalImagesPullingTime="5.713195823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:55.798958881 +0000 UTC m=+74.276994306" watchObservedRunningTime="2026-07-14 09:27:55.799902235 +0000 UTC m=+74.277937655" Jul 14 09:27:55.815062 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:55.815021 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kcts2" podStartSLOduration=68.24917109 podStartE2EDuration="1m12.815008345s" totalImagesPullingTime="4.565837255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:27:55.814301284 +0000 UTC m=+74.292336709" watchObservedRunningTime="2026-07-14 09:27:55.815008345 +0000 UTC m=+74.293043768" Jul 14 09:27:59.598867 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.598829 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-95bkx"] Jul 14 09:27:59.601504 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.601482 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.604703 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.604676 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 09:27:59.604892 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.604870 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 09:27:59.605069 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.604757 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 09:27:59.605147 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.604816 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pkgzk\"" Jul 14 09:27:59.605225 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.604739 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 09:27:59.605913 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.605892 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 09:27:59.606107 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.606090 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 09:27:59.745787 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745757 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.745961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745801 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-wtmp\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.745961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745830 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-root\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.745961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-metrics-client-ca\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.745961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745918 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d74l\" (UniqueName: \"kubernetes.io/projected/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-kube-api-access-2d74l\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.745961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-sys\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.746260 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.745977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-tls\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.746260 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.746049 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-textfile\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.746260 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.746090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847432 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-wtmp\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847432 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847437 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-root\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847475 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-metrics-client-ca\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847498 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d74l\" (UniqueName: \"kubernetes.io/projected/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-kube-api-access-2d74l\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847525 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-sys\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847543 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-tls\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847555 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-root\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847573 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-textfile\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847595 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-wtmp\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.847677 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847625 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-sys\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.848102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.848102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.847967 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-textfile\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.848251 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.848230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-metrics-client-ca\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.848310 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.848246 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.850093 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.850037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-tls\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.850093 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.850037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.856245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.856222 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d74l\" (UniqueName: \"kubernetes.io/projected/faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a-kube-api-access-2d74l\") pod \"node-exporter-95bkx\" (UID: \"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a\") " pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.914294 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:27:59.914266 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-95bkx" Jul 14 09:27:59.928419 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:27:59.928396 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa9d3eb_0a51_4835_ac08_58fcc5d0fe0a.slice/crio-d5dc49f41fc2fd6195307da664ddbaa5a4ce28e7d2d793cb61a9f1236ff52424 WatchSource:0}: Error finding container d5dc49f41fc2fd6195307da664ddbaa5a4ce28e7d2d793cb61a9f1236ff52424: Status 404 returned error can't find the container with id d5dc49f41fc2fd6195307da664ddbaa5a4ce28e7d2d793cb61a9f1236ff52424 Jul 14 09:28:00.799960 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:00.799909 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bkx" event={"ID":"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a","Type":"ContainerStarted","Data":"d5dc49f41fc2fd6195307da664ddbaa5a4ce28e7d2d793cb61a9f1236ff52424"} Jul 14 09:28:01.804936 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:01.804908 2576 generic.go:356] "Generic (PLEG): container finished" podID="faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a" containerID="c5118297b9d88c976a010b405fbad67fc724281fe915b7e9d27f17bdc30a5276" exitCode=0 Jul 14 09:28:01.804936 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:01.804942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bkx" event={"ID":"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a","Type":"ContainerDied","Data":"c5118297b9d88c976a010b405fbad67fc724281fe915b7e9d27f17bdc30a5276"} Jul 14 09:28:02.811244 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:02.811214 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bkx" event={"ID":"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a","Type":"ContainerStarted","Data":"979b6948fe71321467d5cd0d98190ba8d6f7e61249f536cfb5930dce04de7d23"} Jul 14 09:28:02.811610 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:02.811266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-95bkx" event={"ID":"faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a","Type":"ContainerStarted","Data":"d0ff3b9b4e708b7f8090490b87a7432d831dc08fff51809b53c6629f31fadff7"} Jul 14 09:28:02.832851 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:02.832802 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-95bkx" podStartSLOduration=2.804837986 podStartE2EDuration="3.832787309s" totalImagesPullingTime="965.698367ms" totalInitContainerRuntime="62.250956ms" isStatefulPod=true podCreationTimestamp="2026-07-14 09:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:28:02.831098039 +0000 UTC m=+81.309133504" watchObservedRunningTime="2026-07-14 09:28:02.832787309 +0000 UTC m=+81.310822778" Jul 14 09:28:04.468534 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:04.468429 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:28:07.721388 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:07.721359 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c647c656d-c22xh" Jul 14 09:28:09.482679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.482634 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerName="registry" containerID="cri-o://b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5" gracePeriod=30 Jul 14 09:28:09.729929 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.729907 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:28:09.818043 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.817981 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818043 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818014 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818043 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818035 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrmpp\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818106 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818138 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818218 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818253 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted\") pod \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\" (UID: \"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35\") " Jul 14 09:28:09.818567 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818529 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:28:09.818664 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.818634 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 09:28:09.820616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.820583 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:09.820742 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.820712 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:09.820802 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.820728 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:28:09.820840 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.820793 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp" (OuterVolumeSpecName: "kube-api-access-vrmpp") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "kube-api-access-vrmpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:28:09.820840 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.820804 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:28:09.827196 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.827162 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" (UID: "a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 09:28:09.834615 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.834594 2576 generic.go:356] "Generic (PLEG): container finished" podID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerID="b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5" exitCode=0 Jul 14 09:28:09.834693 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.834636 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" Jul 14 09:28:09.834740 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.834634 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" event={"ID":"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35","Type":"ContainerDied","Data":"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5"} Jul 14 09:28:09.834777 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.834758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56ccbcdf69-4bqhq" event={"ID":"a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35","Type":"ContainerDied","Data":"173eed7f847dac8618449cc067708ef7acc6f2aafca7904acde7e3d05d39f57f"} Jul 14 09:28:09.834820 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.834791 2576 scope.go:117] "RemoveContainer" containerID="b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5" Jul 14 09:28:09.843222 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.843207 2576 scope.go:117] "RemoveContainer" containerID="b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5" Jul 14 09:28:09.843502 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:28:09.843476 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5\": container with ID starting with b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5 not found: ID does not exist" containerID="b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5" Jul 14 09:28:09.843559 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.843512 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5"} err="failed to get container status \"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5\": rpc error: code = NotFound desc = could not find container \"b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5\": container with ID starting with b4f4dc9c119e71f229e7cde2aa27210112a5f78c222d5414bf676095f23a38a5 not found: ID does not exist" Jul 14 09:28:09.854438 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.854419 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:28:09.858336 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.858316 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56ccbcdf69-4bqhq"] Jul 14 09:28:09.918874 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918852 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrmpp\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-kube-api-access-vrmpp\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918874 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918872 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-certificates\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918882 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-registry-tls\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918891 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-installation-pull-secrets\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918899 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-ca-trust-extracted\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918906 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-trusted-ca\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918915 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-image-registry-private-configuration\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:09.918976 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:09.918923 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35-bound-sa-token\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:28:10.441316 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:10.441290 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" path="/var/lib/kubelet/pods/a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35/volumes" Jul 14 09:28:26.786696 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:26.786664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kcts2" Jul 14 09:28:35.023995 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:28:35.023964 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8vwt_d39ef447-4a78-4e92-92a2-4f0252a4db58/serve-healthcheck-canary/0.log" Jul 14 09:31:33.149126 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.149092 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr"] Jul 14 09:31:33.149557 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.149364 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerName="registry" Jul 14 09:31:33.149557 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.149376 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerName="registry" Jul 14 09:31:33.149557 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.149416 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c4e1a0-d9d4-46cf-837c-cd4fc13d3b35" containerName="registry" Jul 14 09:31:33.151287 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.151270 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.154007 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.153984 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 09:31:33.154126 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.154104 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 09:31:33.154654 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.154631 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vjhmj\"" Jul 14 09:31:33.154773 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.154698 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 09:31:33.154773 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.154744 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 09:31:33.168299 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.168264 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr"] Jul 14 09:31:33.184005 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.183983 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.184093 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.184014 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.184093 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.184042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.184093 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.184073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktxfm\" (UniqueName: \"kubernetes.io/projected/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-kube-api-access-ktxfm\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.184219 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.184105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.284874 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.284841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.285003 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.284884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.285003 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.284924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.285003 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.284948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktxfm\" (UniqueName: \"kubernetes.io/projected/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-kube-api-access-ktxfm\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.285003 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.284974 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.285405 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.285383 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-manifests\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.287094 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.287073 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-tmp\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.287406 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.287388 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.287453 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.287420 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.293838 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.293813 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktxfm\" (UniqueName: \"kubernetes.io/projected/c5e7119a-1eb7-4565-bc0b-93743d28fa2b-kube-api-access-ktxfm\") pod \"opendatahub-operator-controller-manager-644f4b6bc9-hskvr\" (UID: \"c5e7119a-1eb7-4565-bc0b-93743d28fa2b\") " pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.460996 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.460962 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:33.604616 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:33.604512 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr"] Jul 14 09:31:33.607219 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:31:33.607177 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e7119a_1eb7_4565_bc0b_93743d28fa2b.slice/crio-97ea9018c3d77a66d7b9aa73df5d62426c243ab201be98dd25f2d35932a330ec WatchSource:0}: Error finding container 97ea9018c3d77a66d7b9aa73df5d62426c243ab201be98dd25f2d35932a330ec: Status 404 returned error can't find the container with id 97ea9018c3d77a66d7b9aa73df5d62426c243ab201be98dd25f2d35932a330ec Jul 14 09:31:34.505676 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:34.505637 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" event={"ID":"c5e7119a-1eb7-4565-bc0b-93743d28fa2b","Type":"ContainerStarted","Data":"97ea9018c3d77a66d7b9aa73df5d62426c243ab201be98dd25f2d35932a330ec"} Jul 14 09:31:37.518100 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:37.518056 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" event={"ID":"c5e7119a-1eb7-4565-bc0b-93743d28fa2b","Type":"ContainerStarted","Data":"f713caf30ae5749beda3a08f7b32bffc30ce5e9fdcf011b43e8392e47c57232b"} Jul 14 09:31:38.522749 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:38.522716 2576 generic.go:356] "Generic (PLEG): container finished" podID="c5e7119a-1eb7-4565-bc0b-93743d28fa2b" containerID="f713caf30ae5749beda3a08f7b32bffc30ce5e9fdcf011b43e8392e47c57232b" exitCode=0 Jul 14 09:31:38.523129 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:38.522779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" event={"ID":"c5e7119a-1eb7-4565-bc0b-93743d28fa2b","Type":"ContainerDied","Data":"f713caf30ae5749beda3a08f7b32bffc30ce5e9fdcf011b43e8392e47c57232b"} Jul 14 09:31:39.528327 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:39.528290 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" event={"ID":"c5e7119a-1eb7-4565-bc0b-93743d28fa2b","Type":"ContainerStarted","Data":"769990d7d050de4d82ce1e0193cf4f9d09f021b6cb37a6236d66c2c60e08b537"} Jul 14 09:31:39.528688 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:39.528411 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:39.573023 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:39.572977 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" podStartSLOduration=2.276860768 podStartE2EDuration="6.572963689s" totalImagesPullingTime="3.260928238s" totalInitContainerRuntime="1.035174683s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:39.55934523 +0000 UTC m=+298.037380653" watchObservedRunningTime="2026-07-14 09:31:39.572963689 +0000 UTC m=+298.050999113" Jul 14 09:31:41.908297 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:41.908262 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:31:41.911854 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:41.911831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:31:41.915869 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:41.915486 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 09:31:50.533519 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:50.533489 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-644f4b6bc9-hskvr" Jul 14 09:31:53.060508 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.060474 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw"] Jul 14 09:31:53.062847 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.062831 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.067727 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.067697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 09:31:53.069269 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.068975 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ssjgj\"" Jul 14 09:31:53.069269 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.068996 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 09:31:53.069269 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.069257 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 09:31:53.069772 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.069757 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 09:31:53.114067 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.114047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2efddb-8715-4afe-afe0-c286ca071483-tmp\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.114161 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.114075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.114161 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.114102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6qtj\" (UniqueName: \"kubernetes.io/projected/2f2efddb-8715-4afe-afe0-c286ca071483-kube-api-access-f6qtj\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.126242 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.126222 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw"] Jul 14 09:31:53.214639 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.214616 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.214770 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.214655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6qtj\" (UniqueName: \"kubernetes.io/projected/2f2efddb-8715-4afe-afe0-c286ca071483-kube-api-access-f6qtj\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.214770 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.214707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2efddb-8715-4afe-afe0-c286ca071483-tmp\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.214853 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:31:53.214771 2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.214853 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:31:53.214846 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs podName:2f2efddb-8715-4afe-afe0-c286ca071483 nodeName:}" failed. No retries permitted until 2026-07-14 09:31:53.714830058 +0000 UTC m=+312.192865459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs") pod "kube-auth-proxy-8569779fcf-bwlnw" (UID: "2f2efddb-8715-4afe-afe0-c286ca071483") : secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.216874 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.216850 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f2efddb-8715-4afe-afe0-c286ca071483-tmp\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.227299 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.227277 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6qtj\" (UniqueName: \"kubernetes.io/projected/2f2efddb-8715-4afe-afe0-c286ca071483-kube-api-access-f6qtj\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.719312 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:53.719277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:53.719483 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:31:53.719427 2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 14 09:31:53.719529 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:31:53.719487 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs podName:2f2efddb-8715-4afe-afe0-c286ca071483 nodeName:}" failed. No retries permitted until 2026-07-14 09:31:54.71947297 +0000 UTC m=+313.197508372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs") pod "kube-auth-proxy-8569779fcf-bwlnw" (UID: "2f2efddb-8715-4afe-afe0-c286ca071483") : secret "kube-auth-proxy-tls" not found Jul 14 09:31:54.320262 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.320227 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qtlgs"] Jul 14 09:31:54.322385 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.322367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.324942 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.324917 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 09:31:54.325049 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.325026 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-slcg9\"" Jul 14 09:31:54.336662 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.336638 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qtlgs"] Jul 14 09:31:54.424779 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.424752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-445zn\" (UniqueName: \"kubernetes.io/projected/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-kube-api-access-445zn\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.424962 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.424798 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-tls-certs\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.525412 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.525382 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-tls-certs\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.525525 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.525449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-445zn\" (UniqueName: \"kubernetes.io/projected/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-kube-api-access-445zn\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.527803 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.527781 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-tls-certs\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.534669 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.534646 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-445zn\" (UniqueName: \"kubernetes.io/projected/23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75-kube-api-access-445zn\") pod \"model-serving-api-54d89bd5fc-qtlgs\" (UID: \"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75\") " pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.631936 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.631873 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:54.726574 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.726537 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:54.729143 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.729113 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f2efddb-8715-4afe-afe0-c286ca071483-tls-certs\") pod \"kube-auth-proxy-8569779fcf-bwlnw\" (UID: \"2f2efddb-8715-4afe-afe0-c286ca071483\") " pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:54.757804 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.757779 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qtlgs"] Jul 14 09:31:54.760313 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:31:54.760289 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f88bb4_f8ff_4ea3_b909_6c3c4fec4c75.slice/crio-e8418d6966530877fbde48e34412d679305ce509302cafc6779854bc514e2875 WatchSource:0}: Error finding container e8418d6966530877fbde48e34412d679305ce509302cafc6779854bc514e2875: Status 404 returned error can't find the container with id e8418d6966530877fbde48e34412d679305ce509302cafc6779854bc514e2875 Jul 14 09:31:54.762098 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.762082 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:31:54.872315 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:54.872293 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" Jul 14 09:31:55.000307 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:55.000281 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw"] Jul 14 09:31:55.003159 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:31:55.003133 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2efddb_8715_4afe_afe0_c286ca071483.slice/crio-0cb610ff966867c8ae2d0c6ba01ad468b1688bbf20deb46cb485c5ee4c8897af WatchSource:0}: Error finding container 0cb610ff966867c8ae2d0c6ba01ad468b1688bbf20deb46cb485c5ee4c8897af: Status 404 returned error can't find the container with id 0cb610ff966867c8ae2d0c6ba01ad468b1688bbf20deb46cb485c5ee4c8897af Jul 14 09:31:55.585263 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:55.585221 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" event={"ID":"2f2efddb-8715-4afe-afe0-c286ca071483","Type":"ContainerStarted","Data":"0cb610ff966867c8ae2d0c6ba01ad468b1688bbf20deb46cb485c5ee4c8897af"} Jul 14 09:31:55.586357 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:55.586332 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" event={"ID":"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75","Type":"ContainerStarted","Data":"e8418d6966530877fbde48e34412d679305ce509302cafc6779854bc514e2875"} Jul 14 09:31:57.595601 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:57.595548 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" event={"ID":"23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75","Type":"ContainerStarted","Data":"e23c10275f76d355b5e3818ea217eaaabada2708035b7682cf97eb738c2d80e5"} Jul 14 09:31:57.595995 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:57.595666 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:31:57.614090 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:57.614039 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" podStartSLOduration=1.535979605 podStartE2EDuration="3.614021672s" totalImagesPullingTime="2.078042067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:57.613194657 +0000 UTC m=+316.091230073" watchObservedRunningTime="2026-07-14 09:31:57.614021672 +0000 UTC m=+316.092057097" Jul 14 09:31:58.600664 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:58.600619 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" event={"ID":"2f2efddb-8715-4afe-afe0-c286ca071483","Type":"ContainerStarted","Data":"97707b9e3ff9e4f3bb2600f971e5f307f30457fb002ca9e6b73383ed934bc260"} Jul 14 09:31:58.637802 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:31:58.637759 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8569779fcf-bwlnw" podStartSLOduration=2.729245828 podStartE2EDuration="5.637743399s" totalImagesPullingTime="2.908497571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:31:58.637357841 +0000 UTC m=+317.115393266" watchObservedRunningTime="2026-07-14 09:31:58.637743399 +0000 UTC m=+317.115778823" Jul 14 09:32:02.606837 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.606800 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk"] Jul 14 09:32:02.610045 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.610019 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.620695 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.620670 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk"] Jul 14 09:32:02.627559 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.627536 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 09:32:02.627679 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.627540 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 09:32:02.627731 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.627708 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 09:32:02.628983 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.628967 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-glml4\"" Jul 14 09:32:02.629076 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.628998 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 09:32:02.629200 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.629170 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 09:32:02.682918 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.682884 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-metrics-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.682918 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.682924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbl9v\" (UniqueName: \"kubernetes.io/projected/4a166add-ba0d-433b-a1be-1d0a8f5f7766-kube-api-access-qbl9v\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.683104 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.682953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.683104 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.683073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4a166add-ba0d-433b-a1be-1d0a8f5f7766-manager-config\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.784109 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.784071 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4a166add-ba0d-433b-a1be-1d0a8f5f7766-manager-config\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.784274 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.784135 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-metrics-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.784274 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.784164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbl9v\" (UniqueName: \"kubernetes.io/projected/4a166add-ba0d-433b-a1be-1d0a8f5f7766-kube-api-access-qbl9v\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.784274 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.784208 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.785016 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.784991 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4a166add-ba0d-433b-a1be-1d0a8f5f7766-manager-config\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.786688 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.786665 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.786811 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.786791 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a166add-ba0d-433b-a1be-1d0a8f5f7766-metrics-cert\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.805061 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.805037 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbl9v\" (UniqueName: \"kubernetes.io/projected/4a166add-ba0d-433b-a1be-1d0a8f5f7766-kube-api-access-qbl9v\") pod \"lws-controller-manager-56954f6f7d-498qk\" (UID: \"4a166add-ba0d-433b-a1be-1d0a8f5f7766\") " pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:02.922475 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:02.922421 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:03.055851 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:32:03.055819 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a166add_ba0d_433b_a1be_1d0a8f5f7766.slice/crio-de8ed03c0f7cfc1f1f193be8a4e9a584b2aac49cde80d137a6982b2bed6b0e73 WatchSource:0}: Error finding container de8ed03c0f7cfc1f1f193be8a4e9a584b2aac49cde80d137a6982b2bed6b0e73: Status 404 returned error can't find the container with id de8ed03c0f7cfc1f1f193be8a4e9a584b2aac49cde80d137a6982b2bed6b0e73 Jul 14 09:32:03.060391 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:03.060374 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk"] Jul 14 09:32:03.619697 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:03.619661 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" event={"ID":"4a166add-ba0d-433b-a1be-1d0a8f5f7766","Type":"ContainerStarted","Data":"de8ed03c0f7cfc1f1f193be8a4e9a584b2aac49cde80d137a6982b2bed6b0e73"} Jul 14 09:32:06.633690 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:06.633655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" event={"ID":"4a166add-ba0d-433b-a1be-1d0a8f5f7766","Type":"ContainerStarted","Data":"2f8e8d7fc206259c21eae31064fada9a6047546b8de8cec69c3e57c049ca8e95"} Jul 14 09:32:06.634096 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:06.633787 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:32:06.665541 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:06.665495 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" podStartSLOduration=1.8661037729999999 podStartE2EDuration="4.665483602s" totalImagesPullingTime="2.799379829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:32:06.664230178 +0000 UTC m=+325.142265602" watchObservedRunningTime="2026-07-14 09:32:06.665483602 +0000 UTC m=+325.143519027" Jul 14 09:32:08.604370 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:08.604340 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qtlgs" Jul 14 09:32:17.644061 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:32:17.644023 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56954f6f7d-498qk" Jul 14 09:34:27.318361 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.318331 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:27.320466 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.320449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:27.323262 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.323239 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 09:34:27.324502 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.324484 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nm889\"" Jul 14 09:34:27.324601 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.324492 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 09:34:27.329489 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.329468 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:27.379008 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.378984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b59v4\" (UniqueName: \"kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4\") pod \"authorino-58cd955675-t45sm\" (UID: \"dfd028f2-421a-4a32-83b4-7081d042397e\") " pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:27.479421 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.479395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b59v4\" (UniqueName: \"kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4\") pod \"authorino-58cd955675-t45sm\" (UID: \"dfd028f2-421a-4a32-83b4-7081d042397e\") " pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:27.487403 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.487374 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b59v4\" (UniqueName: \"kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4\") pod \"authorino-58cd955675-t45sm\" (UID: \"dfd028f2-421a-4a32-83b4-7081d042397e\") " pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:27.630458 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.630404 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:27.752711 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:27.752687 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:27.754978 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:34:27.754953 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfd028f2_421a_4a32_83b4_7081d042397e.slice/crio-d99ca0a80e80a9fe3101fced37c5d6a8744deff751393d44378555577a9b8d5f WatchSource:0}: Error finding container d99ca0a80e80a9fe3101fced37c5d6a8744deff751393d44378555577a9b8d5f: Status 404 returned error can't find the container with id d99ca0a80e80a9fe3101fced37c5d6a8744deff751393d44378555577a9b8d5f Jul 14 09:34:28.115177 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:28.115138 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t45sm" event={"ID":"dfd028f2-421a-4a32-83b4-7081d042397e","Type":"ContainerStarted","Data":"d99ca0a80e80a9fe3101fced37c5d6a8744deff751393d44378555577a9b8d5f"} Jul 14 09:34:32.131784 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:32.131752 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t45sm" event={"ID":"dfd028f2-421a-4a32-83b4-7081d042397e","Type":"ContainerStarted","Data":"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f"} Jul 14 09:34:32.164934 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:32.164888 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-t45sm" podStartSLOduration=1.7817700840000001 podStartE2EDuration="5.164872402s" totalImagesPullingTime="3.383102318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:34:32.163461038 +0000 UTC m=+470.641496462" watchObservedRunningTime="2026-07-14 09:34:32.164872402 +0000 UTC m=+470.642907827" Jul 14 09:34:32.478731 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:32.478701 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:34.139262 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:34.139227 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-t45sm" podUID="dfd028f2-421a-4a32-83b4-7081d042397e" containerName="authorino" containerID="cri-o://1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f" gracePeriod=30 Jul 14 09:34:34.372719 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:34.372696 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:34.432166 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:34.432144 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b59v4\" (UniqueName: \"kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4\") pod \"dfd028f2-421a-4a32-83b4-7081d042397e\" (UID: \"dfd028f2-421a-4a32-83b4-7081d042397e\") " Jul 14 09:34:34.434131 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:34.434109 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4" (OuterVolumeSpecName: "kube-api-access-b59v4") pod "dfd028f2-421a-4a32-83b4-7081d042397e" (UID: "dfd028f2-421a-4a32-83b4-7081d042397e"). InnerVolumeSpecName "kube-api-access-b59v4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:34:34.533300 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:34.533274 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b59v4\" (UniqueName: \"kubernetes.io/projected/dfd028f2-421a-4a32-83b4-7081d042397e-kube-api-access-b59v4\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:34:35.143875 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.143846 2576 generic.go:356] "Generic (PLEG): container finished" podID="dfd028f2-421a-4a32-83b4-7081d042397e" containerID="1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f" exitCode=0 Jul 14 09:34:35.144277 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.143898 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-t45sm" Jul 14 09:34:35.144277 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.143931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t45sm" event={"ID":"dfd028f2-421a-4a32-83b4-7081d042397e","Type":"ContainerDied","Data":"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f"} Jul 14 09:34:35.144277 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.143979 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-t45sm" event={"ID":"dfd028f2-421a-4a32-83b4-7081d042397e","Type":"ContainerDied","Data":"d99ca0a80e80a9fe3101fced37c5d6a8744deff751393d44378555577a9b8d5f"} Jul 14 09:34:35.144277 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.143999 2576 scope.go:117] "RemoveContainer" containerID="1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f" Jul 14 09:34:35.152047 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.152031 2576 scope.go:117] "RemoveContainer" containerID="1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f" Jul 14 09:34:35.152315 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:34:35.152298 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f\": container with ID starting with 1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f not found: ID does not exist" containerID="1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f" Jul 14 09:34:35.152387 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.152322 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f"} err="failed to get container status \"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f\": rpc error: code = NotFound desc = could not find container \"1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f\": container with ID starting with 1754ba7e3aa521bc61896601aa1ef6bcead97aba4585db084b6fc3503cec8e1f not found: ID does not exist" Jul 14 09:34:35.160734 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.160714 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:35.164300 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:35.164282 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-t45sm"] Jul 14 09:34:36.441020 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:34:36.440986 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfd028f2-421a-4a32-83b4-7081d042397e" path="/var/lib/kubelet/pods/dfd028f2-421a-4a32-83b4-7081d042397e/volumes" Jul 14 09:35:17.449720 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.449679 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:17.450149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.449950 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfd028f2-421a-4a32-83b4-7081d042397e" containerName="authorino" Jul 14 09:35:17.450149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.449960 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd028f2-421a-4a32-83b4-7081d042397e" containerName="authorino" Jul 14 09:35:17.450149 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.450031 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfd028f2-421a-4a32-83b4-7081d042397e" containerName="authorino" Jul 14 09:35:17.451985 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.451968 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.456748 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.456511 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kk6bs\"" Jul 14 09:35:17.458107 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.457962 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 09:35:17.458280 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.458262 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 09:35:17.469915 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.469891 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:17.528690 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.528666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.528788 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.528704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9bl\" (UniqueName: \"kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.629838 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.629811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.629938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.629860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9bl\" (UniqueName: \"kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.632122 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.632101 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.634459 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.634425 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:35:17.636347 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.636330 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.640845 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.640828 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9bl\" (UniqueName: \"kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl\") pod \"maas-controller-674498dcdb-kss7g\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.645480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.645454 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:35:17.730460 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.730407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzxnl\" (UniqueName: \"kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.730553 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.730481 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.769424 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.769402 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:17.831646 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.831618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.831799 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.831669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzxnl\" (UniqueName: \"kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.834397 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.834363 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.841685 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.841661 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzxnl\" (UniqueName: \"kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl\") pod \"maas-controller-794679ff6-4vd2z\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:17.894625 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.894597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:17.897529 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:35:17.897501 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1b865e_f55f_4015_be68_3741d289c1b6.slice/crio-9dc29bdf38e890bd0ac99b9898cb0a00a1132fea77303ff72c6f19a794ec4322 WatchSource:0}: Error finding container 9dc29bdf38e890bd0ac99b9898cb0a00a1132fea77303ff72c6f19a794ec4322: Status 404 returned error can't find the container with id 9dc29bdf38e890bd0ac99b9898cb0a00a1132fea77303ff72c6f19a794ec4322 Jul 14 09:35:17.953667 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:17.953639 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:18.083406 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:18.083381 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:35:18.085652 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:35:18.085624 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc674e4f1_146b_450f_a6ec_0a2344a6bcfc.slice/crio-16fb6de6663a3f072d6eeca0d863facf3825d13799035b373f255cb187eb405d WatchSource:0}: Error finding container 16fb6de6663a3f072d6eeca0d863facf3825d13799035b373f255cb187eb405d: Status 404 returned error can't find the container with id 16fb6de6663a3f072d6eeca0d863facf3825d13799035b373f255cb187eb405d Jul 14 09:35:18.291685 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:18.291607 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-4vd2z" event={"ID":"c674e4f1-146b-450f-a6ec-0a2344a6bcfc","Type":"ContainerStarted","Data":"16fb6de6663a3f072d6eeca0d863facf3825d13799035b373f255cb187eb405d"} Jul 14 09:35:18.292683 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:18.292660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674498dcdb-kss7g" event={"ID":"1d1b865e-f55f-4015-be68-3741d289c1b6","Type":"ContainerStarted","Data":"9dc29bdf38e890bd0ac99b9898cb0a00a1132fea77303ff72c6f19a794ec4322"} Jul 14 09:35:21.308542 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.308511 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-4vd2z" event={"ID":"c674e4f1-146b-450f-a6ec-0a2344a6bcfc","Type":"ContainerStarted","Data":"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9"} Jul 14 09:35:21.308944 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.308619 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:21.309996 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.309973 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674498dcdb-kss7g" event={"ID":"1d1b865e-f55f-4015-be68-3741d289c1b6","Type":"ContainerStarted","Data":"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a"} Jul 14 09:35:21.310124 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.310029 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:21.345250 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.345205 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794679ff6-4vd2z" podStartSLOduration=1.914036362 podStartE2EDuration="4.345175383s" totalImagesPullingTime="2.431139021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:35:21.342916347 +0000 UTC m=+519.820951771" watchObservedRunningTime="2026-07-14 09:35:21.345175383 +0000 UTC m=+519.823210806" Jul 14 09:35:21.375583 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:21.375540 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-674498dcdb-kss7g" podStartSLOduration=1.766337459 podStartE2EDuration="4.375518731s" totalImagesPullingTime="2.609181272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:35:21.374666564 +0000 UTC m=+519.852701987" watchObservedRunningTime="2026-07-14 09:35:21.375518731 +0000 UTC m=+519.853554155" Jul 14 09:35:32.315995 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.315969 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:35:32.316480 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.316035 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:32.376719 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.376687 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:32.376883 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.376847 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-674498dcdb-kss7g" podUID="1d1b865e-f55f-4015-be68-3741d289c1b6" containerName="manager" containerID="cri-o://d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a" gracePeriod=30 Jul 14 09:35:32.615330 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.615304 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:32.740163 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.740135 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt9bl\" (UniqueName: \"kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl\") pod \"1d1b865e-f55f-4015-be68-3741d289c1b6\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " Jul 14 09:35:32.740309 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.740175 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert\") pod \"1d1b865e-f55f-4015-be68-3741d289c1b6\" (UID: \"1d1b865e-f55f-4015-be68-3741d289c1b6\") " Jul 14 09:35:32.742336 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.742304 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1d1b865e-f55f-4015-be68-3741d289c1b6" (UID: "1d1b865e-f55f-4015-be68-3741d289c1b6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:35:32.742336 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.742327 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl" (OuterVolumeSpecName: "kube-api-access-vt9bl") pod "1d1b865e-f55f-4015-be68-3741d289c1b6" (UID: "1d1b865e-f55f-4015-be68-3741d289c1b6"). InnerVolumeSpecName "kube-api-access-vt9bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:35:32.841024 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.840990 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt9bl\" (UniqueName: \"kubernetes.io/projected/1d1b865e-f55f-4015-be68-3741d289c1b6-kube-api-access-vt9bl\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:35:32.841024 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:32.841018 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d1b865e-f55f-4015-be68-3741d289c1b6-webhook-cert\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:35:33.355414 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.355378 2576 generic.go:356] "Generic (PLEG): container finished" podID="1d1b865e-f55f-4015-be68-3741d289c1b6" containerID="d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a" exitCode=0 Jul 14 09:35:33.355858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.355421 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-674498dcdb-kss7g" Jul 14 09:35:33.355858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.355459 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674498dcdb-kss7g" event={"ID":"1d1b865e-f55f-4015-be68-3741d289c1b6","Type":"ContainerDied","Data":"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a"} Jul 14 09:35:33.355858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.355507 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-674498dcdb-kss7g" event={"ID":"1d1b865e-f55f-4015-be68-3741d289c1b6","Type":"ContainerDied","Data":"9dc29bdf38e890bd0ac99b9898cb0a00a1132fea77303ff72c6f19a794ec4322"} Jul 14 09:35:33.355858 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.355537 2576 scope.go:117] "RemoveContainer" containerID="d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a" Jul 14 09:35:33.363662 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.363644 2576 scope.go:117] "RemoveContainer" containerID="d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a" Jul 14 09:35:33.363907 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:35:33.363889 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a\": container with ID starting with d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a not found: ID does not exist" containerID="d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a" Jul 14 09:35:33.363956 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.363915 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a"} err="failed to get container status \"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a\": rpc error: code = NotFound desc = could not find container \"d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a\": container with ID starting with d5f5e1ebfd0d3ff6771701ba3f3d88ce111a61f7883233f962f21c98018ea66a not found: ID does not exist" Jul 14 09:35:33.375882 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.375856 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:33.379631 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:33.379612 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-674498dcdb-kss7g"] Jul 14 09:35:34.440515 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:34.440448 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d1b865e-f55f-4015-be68-3741d289c1b6" path="/var/lib/kubelet/pods/1d1b865e-f55f-4015-be68-3741d289c1b6/volumes" Jul 14 09:35:38.350232 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.350204 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7p2m"] Jul 14 09:35:38.350654 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.350493 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d1b865e-f55f-4015-be68-3741d289c1b6" containerName="manager" Jul 14 09:35:38.350654 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.350505 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d1b865e-f55f-4015-be68-3741d289c1b6" containerName="manager" Jul 14 09:35:38.350654 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.350549 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d1b865e-f55f-4015-be68-3741d289c1b6" containerName="manager" Jul 14 09:35:38.352348 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.352331 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.354827 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.354807 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9x5vz\"" Jul 14 09:35:38.354962 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.354810 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 09:35:38.363444 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.363422 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7p2m"] Jul 14 09:35:38.482557 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.482528 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-924x6\" (UniqueName: \"kubernetes.io/projected/538ec3f8-7f89-4638-a4e8-935003561218-kube-api-access-924x6\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.482687 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.482566 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/538ec3f8-7f89-4638-a4e8-935003561218-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.583876 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.583844 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-924x6\" (UniqueName: \"kubernetes.io/projected/538ec3f8-7f89-4638-a4e8-935003561218-kube-api-access-924x6\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.584019 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.583895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/538ec3f8-7f89-4638-a4e8-935003561218-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.584657 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.584634 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/538ec3f8-7f89-4638-a4e8-935003561218-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.594651 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.594630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-924x6\" (UniqueName: \"kubernetes.io/projected/538ec3f8-7f89-4638-a4e8-935003561218-kube-api-access-924x6\") pod \"payload-processing-f9b8d9b9b-d7p2m\" (UID: \"538ec3f8-7f89-4638-a4e8-935003561218\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.662827 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.662768 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:38.787725 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:38.787696 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-d7p2m"] Jul 14 09:35:38.790407 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:35:38.790378 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538ec3f8_7f89_4638_a4e8_935003561218.slice/crio-fd82888d8702680db9732a1cedda73462065e64a17af1a1496c70c8f7acf0b69 WatchSource:0}: Error finding container fd82888d8702680db9732a1cedda73462065e64a17af1a1496c70c8f7acf0b69: Status 404 returned error can't find the container with id fd82888d8702680db9732a1cedda73462065e64a17af1a1496c70c8f7acf0b69 Jul 14 09:35:39.381540 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:39.381509 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" event={"ID":"538ec3f8-7f89-4638-a4e8-935003561218","Type":"ContainerStarted","Data":"fd82888d8702680db9732a1cedda73462065e64a17af1a1496c70c8f7acf0b69"} Jul 14 09:35:41.390019 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:41.389939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" event={"ID":"538ec3f8-7f89-4638-a4e8-935003561218","Type":"ContainerStarted","Data":"6147c137c106dea0910185fafbc8af0c2f5a80ca0dc997609dc29eb7187042f8"} Jul 14 09:35:41.390399 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:41.390071 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:35:41.408746 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:41.408695 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" podStartSLOduration=1.161053721 podStartE2EDuration="3.408681043s" totalImagesPullingTime="2.247627322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:35:41.407029829 +0000 UTC m=+539.885065252" watchObservedRunningTime="2026-07-14 09:35:41.408681043 +0000 UTC m=+539.886716467" Jul 14 09:35:52.396173 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:35:52.396140 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-d7p2m" Jul 14 09:36:10.916053 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.916015 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88"] Jul 14 09:36:10.918831 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.918808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:10.921678 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.921658 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 09:36:10.922894 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.922875 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 09:36:10.922993 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.922879 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 09:36:10.922993 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.922887 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zfwqb\"" Jul 14 09:36:10.931564 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:10.931545 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88"] Jul 14 09:36:11.017381 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017353 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.017495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjw5\" (UniqueName: \"kubernetes.io/projected/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-kube-api-access-jcjw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.017495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.017495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.017495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.017495 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.017470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118625 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118598 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjw5\" (UniqueName: \"kubernetes.io/projected/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-kube-api-access-jcjw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118726 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118726 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118655 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118796 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118773 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118875 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.118937 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.118915 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.119044 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.119022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.119079 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.119047 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.119245 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.119220 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.120961 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.120933 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.121162 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.121145 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.127049 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.127026 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjw5\" (UniqueName: \"kubernetes.io/projected/39c8f616-c1ba-4cfa-b3a2-5b440ec865f8-kube-api-access-jcjw5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qzd88\" (UID: \"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.229219 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.229177 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:11.365268 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.365216 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88"] Jul 14 09:36:11.367768 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:36:11.367739 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c8f616_c1ba_4cfa_b3a2_5b440ec865f8.slice/crio-5d1e26e7c8c99cc4a108b0de33c6046066785ab0ca5e22a797ca5708d15b10a8 WatchSource:0}: Error finding container 5d1e26e7c8c99cc4a108b0de33c6046066785ab0ca5e22a797ca5708d15b10a8: Status 404 returned error can't find the container with id 5d1e26e7c8c99cc4a108b0de33c6046066785ab0ca5e22a797ca5708d15b10a8 Jul 14 09:36:11.507421 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:11.507357 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" event={"ID":"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8","Type":"ContainerStarted","Data":"5d1e26e7c8c99cc4a108b0de33c6046066785ab0ca5e22a797ca5708d15b10a8"} Jul 14 09:36:12.512208 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:12.512153 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" event={"ID":"39c8f616-c1ba-4cfa-b3a2-5b440ec865f8","Type":"ContainerStarted","Data":"259ef69014ba71e47bdcd6fa52af4f31da20e565f50297c27f4a7d1ff0d4cd31"} Jul 14 09:36:12.530938 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:12.530899 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" podStartSLOduration=1.5142012 podStartE2EDuration="2.530884516s" totalImagesPullingTime="1.016683316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:36:12.529396923 +0000 UTC m=+571.007432349" watchObservedRunningTime="2026-07-14 09:36:12.530884516 +0000 UTC m=+571.008919940" Jul 14 09:36:21.229338 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:21.229305 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:21.229794 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:21.229362 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:21.241852 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:21.241820 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:21.556094 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:21.556024 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qzd88" Jul 14 09:36:41.614025 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.613993 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm"] Jul 14 09:36:41.617991 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.617955 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.620528 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.620501 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 14 09:36:41.630444 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.630420 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm"] Jul 14 09:36:41.737488 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.737611 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737493 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdhkg\" (UniqueName: \"kubernetes.io/projected/4c460381-d56e-400d-a4ae-5e74a73d7884-kube-api-access-bdhkg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.737611 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737539 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c460381-d56e-400d-a4ae-5e74a73d7884-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.737611 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737563 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.737611 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737587 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.737778 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.737642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838736 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838736 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838912 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838912 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdhkg\" (UniqueName: \"kubernetes.io/projected/4c460381-d56e-400d-a4ae-5e74a73d7884-kube-api-access-bdhkg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838912 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838821 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c460381-d56e-400d-a4ae-5e74a73d7884-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.838912 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.838837 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.839166 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.839139 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.839166 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.839154 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.839286 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.839232 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.841000 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.840975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c460381-d56e-400d-a4ae-5e74a73d7884-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.841259 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.841241 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c460381-d56e-400d-a4ae-5e74a73d7884-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.847623 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.847600 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdhkg\" (UniqueName: \"kubernetes.io/projected/4c460381-d56e-400d-a4ae-5e74a73d7884-kube-api-access-bdhkg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-prvsm\" (UID: \"4c460381-d56e-400d-a4ae-5e74a73d7884\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.929134 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.929115 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:41.933111 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.933092 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:36:41.943060 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:41.943036 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:36:42.058743 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:42.058693 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm"] Jul 14 09:36:42.060798 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:36:42.060772 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c460381_d56e_400d_a4ae_5e74a73d7884.slice/crio-0d21e1751e73174be5c9eeef8ff7bf0e682713662f0a6feecc49b2f2c9ab0121 WatchSource:0}: Error finding container 0d21e1751e73174be5c9eeef8ff7bf0e682713662f0a6feecc49b2f2c9ab0121: Status 404 returned error can't find the container with id 0d21e1751e73174be5c9eeef8ff7bf0e682713662f0a6feecc49b2f2c9ab0121 Jul 14 09:36:42.628949 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:42.628875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" event={"ID":"4c460381-d56e-400d-a4ae-5e74a73d7884","Type":"ContainerStarted","Data":"8725e2ac6685ede5526da4208e99f531f47871d9234605aa6d76ae154143229a"} Jul 14 09:36:42.628949 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:42.628933 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" event={"ID":"4c460381-d56e-400d-a4ae-5e74a73d7884","Type":"ContainerStarted","Data":"0d21e1751e73174be5c9eeef8ff7bf0e682713662f0a6feecc49b2f2c9ab0121"} Jul 14 09:36:42.647894 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:42.647847 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" podStartSLOduration=1.457114883 podStartE2EDuration="1.647832561s" totalImagesPullingTime="190.717678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:36:42.646140711 +0000 UTC m=+601.124176127" watchObservedRunningTime="2026-07-14 09:36:42.647832561 +0000 UTC m=+601.125867984" Jul 14 09:36:51.929607 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:51.929568 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:51.931490 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:51.929634 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:51.942122 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:51.942100 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:52.678929 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:52.678904 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-prvsm" Jul 14 09:36:56.616548 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.616509 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8"] Jul 14 09:36:56.619264 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.619242 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.622027 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.622007 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 14 09:36:56.633299 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.633277 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8"] Jul 14 09:36:56.745947 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.745921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.746084 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.745954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.746084 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.745973 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.746084 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.745991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cght5\" (UniqueName: \"kubernetes.io/projected/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-kube-api-access-cght5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.746084 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.746075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.746249 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.746103 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847067 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847038 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847117 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cght5\" (UniqueName: \"kubernetes.io/projected/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-kube-api-access-cght5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847224 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847208 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847509 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847488 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847582 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847545 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.847640 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.847592 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.849365 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.849343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.849595 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.849580 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.854998 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.854975 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cght5\" (UniqueName: \"kubernetes.io/projected/bf437432-a6d6-4a22-aeae-45e8edb9d7c3-kube-api-access-cght5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8\" (UID: \"bf437432-a6d6-4a22-aeae-45e8edb9d7c3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:56.928860 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:56.928833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:36:57.060529 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:57.060293 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8"] Jul 14 09:36:57.062604 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:36:57.062557 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf437432_a6d6_4a22_aeae_45e8edb9d7c3.slice/crio-74e54671dc67c3582141934948599d83832d3f83b97bc59cda53593a38c509a5 WatchSource:0}: Error finding container 74e54671dc67c3582141934948599d83832d3f83b97bc59cda53593a38c509a5: Status 404 returned error can't find the container with id 74e54671dc67c3582141934948599d83832d3f83b97bc59cda53593a38c509a5 Jul 14 09:36:57.064045 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:57.064026 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 09:36:57.689955 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:57.689918 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" event={"ID":"bf437432-a6d6-4a22-aeae-45e8edb9d7c3","Type":"ContainerStarted","Data":"fd75d83e29d69df87333e73ec91353f3994d57b867ed631b576da33ed7a906bc"} Jul 14 09:36:57.689955 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:57.689959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" event={"ID":"bf437432-a6d6-4a22-aeae-45e8edb9d7c3","Type":"ContainerStarted","Data":"74e54671dc67c3582141934948599d83832d3f83b97bc59cda53593a38c509a5"} Jul 14 09:36:57.707965 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:36:57.707918 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" podStartSLOduration=1.5064640969999998 podStartE2EDuration="1.707905118s" totalImagesPullingTime="201.441021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:36:57.706545967 +0000 UTC m=+616.184581415" watchObservedRunningTime="2026-07-14 09:36:57.707905118 +0000 UTC m=+616.185940542" Jul 14 09:37:06.929675 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:37:06.929593 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:37:06.929675 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:37:06.929641 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:37:06.942063 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:37:06.942039 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:37:07.745076 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:37:07.745044 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8" Jul 14 09:38:28.681712 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.681681 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-695c4bb98d-6fcn5"] Jul 14 09:38:28.683818 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.683800 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.686423 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.686397 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 09:38:28.686531 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.686402 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 09:38:28.687587 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.687535 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 09:38:28.687676 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.687616 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nm889\"" Jul 14 09:38:28.692389 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.692370 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-695c4bb98d-6fcn5"] Jul 14 09:38:28.824372 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.824346 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-tls-cert\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.824478 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.824397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlm26\" (UniqueName: \"kubernetes.io/projected/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-kube-api-access-wlm26\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.925002 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.924971 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlm26\" (UniqueName: \"kubernetes.io/projected/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-kube-api-access-wlm26\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.925102 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.925029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-tls-cert\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.927688 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.927666 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-tls-cert\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.932563 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.932504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlm26\" (UniqueName: \"kubernetes.io/projected/8a673ea3-65ca-4195-a3ce-d2df44ef0f01-kube-api-access-wlm26\") pod \"authorino-695c4bb98d-6fcn5\" (UID: \"8a673ea3-65ca-4195-a3ce-d2df44ef0f01\") " pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:28.993888 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:28.993858 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-695c4bb98d-6fcn5" Jul 14 09:38:29.123237 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:29.123206 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-695c4bb98d-6fcn5"] Jul 14 09:38:29.126215 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:38:29.126165 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a673ea3_65ca_4195_a3ce_d2df44ef0f01.slice/crio-9f8debc8e6568cc20fca682e1a047df767b4820e02a2e236e6cb4428c660ad22 WatchSource:0}: Error finding container 9f8debc8e6568cc20fca682e1a047df767b4820e02a2e236e6cb4428c660ad22: Status 404 returned error can't find the container with id 9f8debc8e6568cc20fca682e1a047df767b4820e02a2e236e6cb4428c660ad22 Jul 14 09:38:30.098416 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:30.098383 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c4bb98d-6fcn5" event={"ID":"8a673ea3-65ca-4195-a3ce-d2df44ef0f01","Type":"ContainerStarted","Data":"8c88e6e785ff68bd225482e667fc7ca0a611cd6b573bd8ad3996ace6b0fb4d08"} Jul 14 09:38:30.098875 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:30.098426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-695c4bb98d-6fcn5" event={"ID":"8a673ea3-65ca-4195-a3ce-d2df44ef0f01","Type":"ContainerStarted","Data":"9f8debc8e6568cc20fca682e1a047df767b4820e02a2e236e6cb4428c660ad22"} Jul 14 09:38:30.120368 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:38:30.120315 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-695c4bb98d-6fcn5" podStartSLOduration=1.652229161 podStartE2EDuration="2.120295617s" totalImagesPullingTime="468.066456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:38:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:38:30.119058192 +0000 UTC m=+708.597093619" watchObservedRunningTime="2026-07-14 09:38:30.120295617 +0000 UTC m=+708.598331042" Jul 14 09:39:42.604559 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.604490 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:39:42.605062 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.604786 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-794679ff6-4vd2z" podUID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" containerName="manager" containerID="cri-o://4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9" gracePeriod=30 Jul 14 09:39:42.846753 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.846725 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:39:42.996467 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.996441 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert\") pod \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " Jul 14 09:39:42.996604 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.996482 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzxnl\" (UniqueName: \"kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl\") pod \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\" (UID: \"c674e4f1-146b-450f-a6ec-0a2344a6bcfc\") " Jul 14 09:39:42.998449 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.998422 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c674e4f1-146b-450f-a6ec-0a2344a6bcfc" (UID: "c674e4f1-146b-450f-a6ec-0a2344a6bcfc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 09:39:42.998557 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:42.998535 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl" (OuterVolumeSpecName: "kube-api-access-tzxnl") pod "c674e4f1-146b-450f-a6ec-0a2344a6bcfc" (UID: "c674e4f1-146b-450f-a6ec-0a2344a6bcfc"). InnerVolumeSpecName "kube-api-access-tzxnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 09:39:43.097082 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.097055 2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-webhook-cert\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:39:43.097082 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.097079 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzxnl\" (UniqueName: \"kubernetes.io/projected/c674e4f1-146b-450f-a6ec-0a2344a6bcfc-kube-api-access-tzxnl\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 09:39:43.405010 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.404922 2576 generic.go:356] "Generic (PLEG): container finished" podID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" containerID="4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9" exitCode=0 Jul 14 09:39:43.405010 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.404979 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-4vd2z" Jul 14 09:39:43.405010 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.404991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-4vd2z" event={"ID":"c674e4f1-146b-450f-a6ec-0a2344a6bcfc","Type":"ContainerDied","Data":"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9"} Jul 14 09:39:43.405309 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.405058 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-4vd2z" event={"ID":"c674e4f1-146b-450f-a6ec-0a2344a6bcfc","Type":"ContainerDied","Data":"16fb6de6663a3f072d6eeca0d863facf3825d13799035b373f255cb187eb405d"} Jul 14 09:39:43.405309 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.405084 2576 scope.go:117] "RemoveContainer" containerID="4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9" Jul 14 09:39:43.414628 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.414611 2576 scope.go:117] "RemoveContainer" containerID="4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9" Jul 14 09:39:43.414892 ip-10-0-129-15 kubenswrapper[2576]: E0714 09:39:43.414874 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9\": container with ID starting with 4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9 not found: ID does not exist" containerID="4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9" Jul 14 09:39:43.414946 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.414899 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9"} err="failed to get container status \"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9\": rpc error: code = NotFound desc = could not find container \"4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9\": container with ID starting with 4562d48fa9ee5e1857a40a1c36335cf52413ee8c84cd0ffaf9e79a37d4617aa9 not found: ID does not exist" Jul 14 09:39:43.425769 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.425748 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:39:43.429802 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:43.429782 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-794679ff6-4vd2z"] Jul 14 09:39:44.228999 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.228963 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794679ff6-6kj66"] Jul 14 09:39:44.229366 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.229257 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" containerName="manager" Jul 14 09:39:44.229366 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.229268 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" containerName="manager" Jul 14 09:39:44.229366 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.229325 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" containerName="manager" Jul 14 09:39:44.232095 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.232076 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.234549 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.234531 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 09:39:44.235540 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.235523 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kk6bs\"" Jul 14 09:39:44.235601 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.235553 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 09:39:44.239171 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.238837 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794679ff6-6kj66"] Jul 14 09:39:44.408672 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.408647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-webhook-cert\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.408809 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.408725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bn72\" (UniqueName: \"kubernetes.io/projected/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-kube-api-access-6bn72\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.440888 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.440861 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c674e4f1-146b-450f-a6ec-0a2344a6bcfc" path="/var/lib/kubelet/pods/c674e4f1-146b-450f-a6ec-0a2344a6bcfc/volumes" Jul 14 09:39:44.510076 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.510028 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-webhook-cert\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.510076 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.510060 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bn72\" (UniqueName: \"kubernetes.io/projected/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-kube-api-access-6bn72\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.512413 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.512393 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-webhook-cert\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.518502 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.518480 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bn72\" (UniqueName: \"kubernetes.io/projected/0dc40375-ac01-420e-a388-ff7fdb6f8ef1-kube-api-access-6bn72\") pod \"maas-controller-794679ff6-6kj66\" (UID: \"0dc40375-ac01-420e-a388-ff7fdb6f8ef1\") " pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.543261 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.543239 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:44.663094 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:44.663071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794679ff6-6kj66"] Jul 14 09:39:44.665514 ip-10-0-129-15 kubenswrapper[2576]: W0714 09:39:44.665490 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc40375_ac01_420e_a388_ff7fdb6f8ef1.slice/crio-55ae2455d82b9f9776d547c1ac4cc9d07fb77e0261bd2731844b13fc8df43068 WatchSource:0}: Error finding container 55ae2455d82b9f9776d547c1ac4cc9d07fb77e0261bd2731844b13fc8df43068: Status 404 returned error can't find the container with id 55ae2455d82b9f9776d547c1ac4cc9d07fb77e0261bd2731844b13fc8df43068 Jul 14 09:39:45.415940 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:45.415906 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-6kj66" event={"ID":"0dc40375-ac01-420e-a388-ff7fdb6f8ef1","Type":"ContainerStarted","Data":"51ea6cfc323729f00e276e64a332383457e1cb549b3780d0f600d1f1b8a83036"} Jul 14 09:39:45.416359 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:45.415955 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794679ff6-6kj66" event={"ID":"0dc40375-ac01-420e-a388-ff7fdb6f8ef1","Type":"ContainerStarted","Data":"55ae2455d82b9f9776d547c1ac4cc9d07fb77e0261bd2731844b13fc8df43068"} Jul 14 09:39:45.416359 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:45.415985 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:39:45.432719 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:45.432629 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794679ff6-6kj66" podStartSLOduration=0.926148646 podStartE2EDuration="1.432613166s" totalImagesPullingTime="506.46452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 09:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 09:39:45.430861764 +0000 UTC m=+783.908897190" watchObservedRunningTime="2026-07-14 09:39:45.432613166 +0000 UTC m=+783.910648589" Jul 14 09:39:56.422574 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:39:56.422539 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794679ff6-6kj66" Jul 14 09:41:41.964913 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:41:41.964838 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:41:41.971937 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:41:41.971918 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:46:41.988903 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:46:41.988875 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:46:41.993903 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:46:41.993882 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:51:42.012993 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:51:42.012966 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:51:42.015584 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:51:42.015407 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:56:42.039697 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:56:42.039596 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 09:56:42.046555 ip-10-0-129-15 kubenswrapper[2576]: I0714 09:56:42.040124 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:01:42.065616 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:01:42.065488 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:01:42.068976 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:01:42.066676 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:02:04.926573 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.926538 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:02:04.929861 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.929844 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:04.932595 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.932568 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-06e8a8\"/\"openshift-service-ca.crt\"" Jul 14 10:02:04.933718 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.933697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-06e8a8\"/\"default-dockercfg-x5h45\"" Jul 14 10:02:04.933718 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.933715 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-06e8a8\"/\"kube-root-ca.crt\"" Jul 14 10:02:04.933897 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.933748 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-06e8a8\"/\"test-model-06e8a8-kserve-self-signed-certs\"" Jul 14 10:02:04.939750 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:04.939727 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:02:05.033658 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033633 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.033813 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.033813 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.033813 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.033921 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.033921 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.033875 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p4h6\" (UniqueName: \"kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134390 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p4h6\" (UniqueName: \"kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134467 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134825 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134802 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134877 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.134937 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.134918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.136624 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.136606 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.136889 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.136872 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.153959 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.153930 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p4h6\" (UniqueName: \"kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6\") pod \"test-model-06e8a8-kserve-5745b998d5-mccrf\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.238554 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.238491 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:05.366514 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:05.366490 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:02:05.368595 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:02:05.368572 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd08a10fa_49a7_4711_b5af_e8ccc34ad422.slice/crio-189174b103ae2e76a025b1bbc9c7d3a678ba95865ab6ebf27cce11159edcd69e WatchSource:0}: Error finding container 189174b103ae2e76a025b1bbc9c7d3a678ba95865ab6ebf27cce11159edcd69e: Status 404 returned error can't find the container with id 189174b103ae2e76a025b1bbc9c7d3a678ba95865ab6ebf27cce11159edcd69e Jul 14 10:02:06.021490 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:06.021458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" event={"ID":"d08a10fa-49a7-4711-b5af-e8ccc34ad422","Type":"ContainerStarted","Data":"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375"} Jul 14 10:02:06.021862 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:06.021503 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" event={"ID":"d08a10fa-49a7-4711-b5af-e8ccc34ad422","Type":"ContainerStarted","Data":"189174b103ae2e76a025b1bbc9c7d3a678ba95865ab6ebf27cce11159edcd69e"} Jul 14 10:02:06.040008 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:06.039965 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" podStartSLOduration=2.039951363 podStartE2EDuration="2.039951363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:02:06.037770768 +0000 UTC m=+2124.515806192" watchObservedRunningTime="2026-07-14 10:02:06.039951363 +0000 UTC m=+2124.517986786" Jul 14 10:02:15.239492 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:15.239443 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:15.239492 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:15.239506 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:15.252300 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:15.252271 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:02:16.075373 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:02:16.075343 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:03:28.958415 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:28.958381 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:03:43.978109 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:43.978045 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" podUID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" containerName="main" containerID="cri-o://a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375" gracePeriod=45 Jul 14 10:03:44.229497 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.229435 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:03:44.311898 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.311868 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p4h6\" (UniqueName: \"kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.311910 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.311942 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.311978 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312013 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312034 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home\") pod \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\" (UID: \"d08a10fa-49a7-4711-b5af-e8ccc34ad422\") " Jul 14 10:03:44.312315 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312212 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache" (OuterVolumeSpecName: "model-cache") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:03:44.312315 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312227 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:03:44.312423 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312320 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home" (OuterVolumeSpecName: "home") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:03:44.312423 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312330 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tmp-dir\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.312423 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.312357 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-model-cache\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.314139 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.314112 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm" (OuterVolumeSpecName: "dshm") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:03:44.314139 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.314126 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:03:44.314327 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.314176 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6" (OuterVolumeSpecName: "kube-api-access-5p4h6") pod "d08a10fa-49a7-4711-b5af-e8ccc34ad422" (UID: "d08a10fa-49a7-4711-b5af-e8ccc34ad422"). InnerVolumeSpecName "kube-api-access-5p4h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:03:44.412956 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.412934 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p4h6\" (UniqueName: \"kubernetes.io/projected/d08a10fa-49a7-4711-b5af-e8ccc34ad422-kube-api-access-5p4h6\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.412956 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.412954 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d08a10fa-49a7-4711-b5af-e8ccc34ad422-tls-certs\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.413083 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.412965 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-dshm\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.413083 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.412974 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d08a10fa-49a7-4711-b5af-e8ccc34ad422-home\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:03:44.439375 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.439348 2576 generic.go:356] "Generic (PLEG): container finished" podID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" containerID="a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375" exitCode=0 Jul 14 10:03:44.439475 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.439447 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" Jul 14 10:03:44.441664 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.441633 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" event={"ID":"d08a10fa-49a7-4711-b5af-e8ccc34ad422","Type":"ContainerDied","Data":"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375"} Jul 14 10:03:44.441767 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.441686 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf" event={"ID":"d08a10fa-49a7-4711-b5af-e8ccc34ad422","Type":"ContainerDied","Data":"189174b103ae2e76a025b1bbc9c7d3a678ba95865ab6ebf27cce11159edcd69e"} Jul 14 10:03:44.441767 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.441711 2576 scope.go:117] "RemoveContainer" containerID="a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375" Jul 14 10:03:44.450478 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.450465 2576 scope.go:117] "RemoveContainer" containerID="a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375" Jul 14 10:03:44.450718 ip-10-0-129-15 kubenswrapper[2576]: E0714 10:03:44.450699 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375\": container with ID starting with a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375 not found: ID does not exist" containerID="a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375" Jul 14 10:03:44.450755 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.450732 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375"} err="failed to get container status \"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375\": rpc error: code = NotFound desc = could not find container \"a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375\": container with ID starting with a215ba3a311859d9ed6081ac0b5d8abbf20c86a8af8b4f3eb60ad6f67407a375 not found: ID does not exist" Jul 14 10:03:44.463051 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.463031 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:03:44.467354 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:44.467334 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-06e8a8/test-model-06e8a8-kserve-5745b998d5-mccrf"] Jul 14 10:03:46.441599 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:03:46.441558 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" path="/var/lib/kubelet/pods/d08a10fa-49a7-4711-b5af-e8ccc34ad422/volumes" Jul 14 10:06:07.244371 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.244337 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:06:07.244808 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.244627 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" containerName="main" Jul 14 10:06:07.244808 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.244638 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" containerName="main" Jul 14 10:06:07.244808 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.244687 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08a10fa-49a7-4711-b5af-e8ccc34ad422" containerName="main" Jul 14 10:06:07.247428 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.247408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.251572 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.251549 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"kube-root-ca.crt\"" Jul 14 10:06:07.251689 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.251628 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"auth-test-model-5d32fd-kserve-self-signed-certs\"" Jul 14 10:06:07.251759 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.251691 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"openshift-service-ca.crt\"" Jul 14 10:06:07.251759 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.251709 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"default-dockercfg-pmmcl\"" Jul 14 10:06:07.258019 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.257997 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:06:07.367893 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.367862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.368015 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.367914 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.368015 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.367993 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.368106 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.368024 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.368106 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.368050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.368106 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.368072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxvvj\" (UniqueName: \"kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469377 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469350 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469502 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469502 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469502 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469424 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxvvj\" (UniqueName: \"kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469502 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469462 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469699 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469759 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469733 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469867 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469845 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.469922 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.469868 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.471691 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.471662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.471998 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.471977 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.477780 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.477760 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxvvj\" (UniqueName: \"kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj\") pod \"auth-test-model-5d32fd-kserve-857668cd75-zrwss\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.556907 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.556835 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:07.682065 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:07.682042 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:06:07.685464 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:06:07.685346 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6763e17_f521_4b4f_9bac_deacf2ca334e.slice/crio-733a115e0c4ea2fea06aa3a8e67df3f388ccccc55fe8c576324827ed86a3e9c1 WatchSource:0}: Error finding container 733a115e0c4ea2fea06aa3a8e67df3f388ccccc55fe8c576324827ed86a3e9c1: Status 404 returned error can't find the container with id 733a115e0c4ea2fea06aa3a8e67df3f388ccccc55fe8c576324827ed86a3e9c1 Jul 14 10:06:08.043238 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:08.043172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" event={"ID":"e6763e17-f521-4b4f-9bac-deacf2ca334e","Type":"ContainerStarted","Data":"302667ed4e920aed458535fc6e8f206fdf72f73f9dd733c32999d103b85113e1"} Jul 14 10:06:08.043393 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:08.043250 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" event={"ID":"e6763e17-f521-4b4f-9bac-deacf2ca334e","Type":"ContainerStarted","Data":"733a115e0c4ea2fea06aa3a8e67df3f388ccccc55fe8c576324827ed86a3e9c1"} Jul 14 10:06:08.064581 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:08.064523 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" podStartSLOduration=1.064505621 podStartE2EDuration="1.064505621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:08.062645445 +0000 UTC m=+2366.540680870" watchObservedRunningTime="2026-07-14 10:06:08.064505621 +0000 UTC m=+2366.542541101" Jul 14 10:06:17.557500 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:17.557456 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:17.557923 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:17.557523 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:17.570157 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:17.570136 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:18.101137 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:18.101109 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:06:42.089486 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:42.089380 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:06:42.092003 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:06:42.091980 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:08:44.613171 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:44.613089 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:08:59.632345 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.632286 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" podUID="e6763e17-f521-4b4f-9bac-deacf2ca334e" containerName="main" containerID="cri-o://302667ed4e920aed458535fc6e8f206fdf72f73f9dd733c32999d103b85113e1" gracePeriod=45 Jul 14 10:08:59.790908 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.790874 2576 generic.go:356] "Generic (PLEG): container finished" podID="e6763e17-f521-4b4f-9bac-deacf2ca334e" containerID="302667ed4e920aed458535fc6e8f206fdf72f73f9dd733c32999d103b85113e1" exitCode=0 Jul 14 10:08:59.791104 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.790921 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" event={"ID":"e6763e17-f521-4b4f-9bac-deacf2ca334e","Type":"ContainerDied","Data":"302667ed4e920aed458535fc6e8f206fdf72f73f9dd733c32999d103b85113e1"} Jul 14 10:08:59.877337 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.877315 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:08:59.951674 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951650 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.951799 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951689 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.951799 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951724 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.951799 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951746 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxvvj\" (UniqueName: \"kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.951799 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951784 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.952088 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951839 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home\") pod \"e6763e17-f521-4b4f-9bac-deacf2ca334e\" (UID: \"e6763e17-f521-4b4f-9bac-deacf2ca334e\") " Jul 14 10:08:59.952088 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.951982 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:59.952212 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.952076 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:59.952212 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.952152 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home" (OuterVolumeSpecName: "home") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:59.953853 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.953832 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm" (OuterVolumeSpecName: "dshm") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:59.953945 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.953922 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:08:59.954000 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:08:59.953982 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj" (OuterVolumeSpecName: "kube-api-access-hxvvj") pod "e6763e17-f521-4b4f-9bac-deacf2ca334e" (UID: "e6763e17-f521-4b4f-9bac-deacf2ca334e"). InnerVolumeSpecName "kube-api-access-hxvvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:09:00.053028 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.052996 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-model-cache\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.053028 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.053028 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-home\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.053028 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.053037 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6763e17-f521-4b4f-9bac-deacf2ca334e-tls-certs\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.053202 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.053046 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-dshm\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.053202 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.053053 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6763e17-f521-4b4f-9bac-deacf2ca334e-tmp-dir\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.053202 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.053061 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxvvj\" (UniqueName: \"kubernetes.io/projected/e6763e17-f521-4b4f-9bac-deacf2ca334e-kube-api-access-hxvvj\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.797367 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.797266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" event={"ID":"e6763e17-f521-4b4f-9bac-deacf2ca334e","Type":"ContainerDied","Data":"733a115e0c4ea2fea06aa3a8e67df3f388ccccc55fe8c576324827ed86a3e9c1"} Jul 14 10:09:00.797367 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.797308 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss" Jul 14 10:09:00.797367 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.797339 2576 scope.go:117] "RemoveContainer" containerID="302667ed4e920aed458535fc6e8f206fdf72f73f9dd733c32999d103b85113e1" Jul 14 10:09:00.814572 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.814533 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:09:00.818640 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:00.818603 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/auth-test-model-5d32fd-kserve-857668cd75-zrwss"] Jul 14 10:09:02.443290 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:02.443256 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6763e17-f521-4b4f-9bac-deacf2ca334e" path="/var/lib/kubelet/pods/e6763e17-f521-4b4f-9bac-deacf2ca334e/volumes" Jul 14 10:09:08.143870 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.143827 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:09:08.144355 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.144306 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6763e17-f521-4b4f-9bac-deacf2ca334e" containerName="main" Jul 14 10:09:08.144355 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.144326 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6763e17-f521-4b4f-9bac-deacf2ca334e" containerName="main" Jul 14 10:09:08.144475 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.144432 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6763e17-f521-4b4f-9bac-deacf2ca334e" containerName="main" Jul 14 10:09:08.148675 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.148651 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.152975 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.152939 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"openshift-service-ca.crt\"" Jul 14 10:09:08.153120 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.153082 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"default-dockercfg-pmmcl\"" Jul 14 10:09:08.154164 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.154142 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"kube-root-ca.crt\"" Jul 14 10:09:08.154267 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.154176 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5d32fd\"/\"sub-test-model-5d32fd-kserve-self-signed-certs\"" Jul 14 10:09:08.162082 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.162056 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:09:08.208157 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208131 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.208279 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208174 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.208279 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208242 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.208279 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.208437 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntww\" (UniqueName: \"kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.208437 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.208352 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309373 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309352 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309535 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309535 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309535 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309535 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntww\" (UniqueName: \"kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309535 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309825 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309801 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309884 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.309884 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.309868 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.311812 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.311792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.312108 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.312088 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.317541 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.317518 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntww\" (UniqueName: \"kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww\") pod \"sub-test-model-5d32fd-kserve-675d567899-bmkqd\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.457095 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.457072 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:08.583635 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.583609 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:09:08.586046 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:09:08.586020 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2fe19b2_ee14_4c0c_a55a_9c711e52cffa.slice/crio-048d2eb2c7f2927bcf87623dd11126dc6e17eabc095346c6a34c0a628bafb2c7 WatchSource:0}: Error finding container 048d2eb2c7f2927bcf87623dd11126dc6e17eabc095346c6a34c0a628bafb2c7: Status 404 returned error can't find the container with id 048d2eb2c7f2927bcf87623dd11126dc6e17eabc095346c6a34c0a628bafb2c7 Jul 14 10:09:08.834938 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.834846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" event={"ID":"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa","Type":"ContainerStarted","Data":"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f"} Jul 14 10:09:08.834938 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.834905 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" event={"ID":"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa","Type":"ContainerStarted","Data":"048d2eb2c7f2927bcf87623dd11126dc6e17eabc095346c6a34c0a628bafb2c7"} Jul 14 10:09:08.854195 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:08.854140 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" podStartSLOduration=0.854124262 podStartE2EDuration="854.124262ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:09:08.853342664 +0000 UTC m=+2547.331378088" watchObservedRunningTime="2026-07-14 10:09:08.854124262 +0000 UTC m=+2547.332159685" Jul 14 10:09:18.457432 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:18.457392 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:18.457867 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:18.457477 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:18.470008 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:18.469973 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:09:18.892072 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:09:18.891986 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:10:18.192079 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:18.192007 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:10:33.211498 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.211435 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" podUID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" containerName="main" containerID="cri-o://3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f" gracePeriod=45 Jul 14 10:10:33.458435 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.458398 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:10:33.542508 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542436 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542508 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542477 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542508 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542501 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pntww\" (UniqueName: \"kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542760 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542544 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542760 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542566 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542760 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542580 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home\") pod \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\" (UID: \"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa\") " Jul 14 10:10:33.542760 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542732 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:10:33.542937 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542828 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:10:33.542996 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.542939 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home" (OuterVolumeSpecName: "home") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:10:33.544557 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.544531 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:10:33.544651 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.544573 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww" (OuterVolumeSpecName: "kube-api-access-pntww") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "kube-api-access-pntww". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:10:33.544651 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.544575 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm" (OuterVolumeSpecName: "dshm") pod "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" (UID: "f2fe19b2-ee14-4c0c-a55a-9c711e52cffa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:10:33.643164 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643122 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pntww\" (UniqueName: \"kubernetes.io/projected/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-kube-api-access-pntww\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:33.643164 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643159 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-model-cache\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:33.643164 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643170 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-dshm\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:33.643385 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643177 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-home\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:33.643385 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643207 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tls-certs\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:33.643385 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:33.643215 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa-tmp-dir\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:10:34.206253 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.206221 2576 generic.go:356] "Generic (PLEG): container finished" podID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" containerID="3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f" exitCode=0 Jul 14 10:10:34.206432 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.206266 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" Jul 14 10:10:34.206432 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.206296 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" event={"ID":"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa","Type":"ContainerDied","Data":"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f"} Jul 14 10:10:34.206432 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.206328 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd" event={"ID":"f2fe19b2-ee14-4c0c-a55a-9c711e52cffa","Type":"ContainerDied","Data":"048d2eb2c7f2927bcf87623dd11126dc6e17eabc095346c6a34c0a628bafb2c7"} Jul 14 10:10:34.206432 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.206344 2576 scope.go:117] "RemoveContainer" containerID="3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f" Jul 14 10:10:34.215090 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.214893 2576 scope.go:117] "RemoveContainer" containerID="3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f" Jul 14 10:10:34.215404 ip-10-0-129-15 kubenswrapper[2576]: E0714 10:10:34.215159 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f\": container with ID starting with 3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f not found: ID does not exist" containerID="3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f" Jul 14 10:10:34.215404 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.215221 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f"} err="failed to get container status \"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f\": rpc error: code = NotFound desc = could not find container \"3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f\": container with ID starting with 3f1e46ad483a1b022e74f795639b4682db20547b09f30b9977677174711c6e6f not found: ID does not exist" Jul 14 10:10:34.228403 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.228380 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:10:34.231630 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.231606 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5d32fd/sub-test-model-5d32fd-kserve-675d567899-bmkqd"] Jul 14 10:10:34.442156 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:34.442128 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" path="/var/lib/kubelet/pods/f2fe19b2-ee14-4c0c-a55a-9c711e52cffa/volumes" Jul 14 10:10:49.041675 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.041640 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:10:49.042103 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.041957 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" containerName="main" Jul 14 10:10:49.042103 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.041970 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" containerName="main" Jul 14 10:10:49.042103 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.042024 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2fe19b2-ee14-4c0c-a55a-9c711e52cffa" containerName="main" Jul 14 10:10:49.046220 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.046203 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.049121 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.049096 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ad5ead\"/\"kube-root-ca.crt\"" Jul 14 10:10:49.049266 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.049248 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ad5ead\"/\"rate-test-model-ad5ead-kserve-self-signed-certs\"" Jul 14 10:10:49.050513 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.050493 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ad5ead\"/\"openshift-service-ca.crt\"" Jul 14 10:10:49.050588 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.050497 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ad5ead\"/\"default-dockercfg-gz8nm\"" Jul 14 10:10:49.056820 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.056798 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:10:49.168585 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168557 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.168698 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168587 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.168698 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.168698 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168644 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngnnm\" (UniqueName: \"kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.168698 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168687 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.168829 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.168718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269070 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269042 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269217 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269079 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269217 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269217 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269217 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269157 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngnnm\" (UniqueName: \"kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269217 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269198 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269529 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269504 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269600 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269581 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.269639 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.269627 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.271467 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.271445 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.271580 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.271561 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.279203 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.279164 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngnnm\" (UniqueName: \"kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm\") pod \"rate-test-model-ad5ead-kserve-5b78787699-llf8p\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.355277 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.355221 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:49.488696 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:49.488672 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:10:49.490360 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:10:49.490324 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9de114cc_5e0c_4985_8bb6_36cce96ae58d.slice/crio-c35416892d05f9836ab65c5dae127423b923bfd33320392a827c687e3a9d7c26 WatchSource:0}: Error finding container c35416892d05f9836ab65c5dae127423b923bfd33320392a827c687e3a9d7c26: Status 404 returned error can't find the container with id c35416892d05f9836ab65c5dae127423b923bfd33320392a827c687e3a9d7c26 Jul 14 10:10:50.275029 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:50.274981 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" event={"ID":"9de114cc-5e0c-4985-8bb6-36cce96ae58d","Type":"ContainerStarted","Data":"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57"} Jul 14 10:10:50.275455 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:50.275044 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" event={"ID":"9de114cc-5e0c-4985-8bb6-36cce96ae58d","Type":"ContainerStarted","Data":"c35416892d05f9836ab65c5dae127423b923bfd33320392a827c687e3a9d7c26"} Jul 14 10:10:50.295658 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:50.295607 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" podStartSLOduration=1.295587202 podStartE2EDuration="1.295587202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:10:50.294331272 +0000 UTC m=+2648.772366696" watchObservedRunningTime="2026-07-14 10:10:50.295587202 +0000 UTC m=+2648.773622690" Jul 14 10:10:59.355876 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:59.355836 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:59.356291 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:59.355925 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:10:59.368685 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:10:59.368665 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:11:00.328935 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:00.328904 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:11:33.629304 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:33.629267 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:11:34.682740 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.682661 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7151-139766694350896"] Jul 14 10:11:34.685827 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.685808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:34.688351 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.688323 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-zhv22\"" Jul 14 10:11:34.691850 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.691824 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766694350896"] Jul 14 10:11:34.818665 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.818641 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsg2h\" (UniqueName: \"kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h\") pod \"test-curl-7151-139766694350896\" (UID: \"6a595e79-7e28-4b87-aa40-1b33357fcb3a\") " pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:34.919760 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.919730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsg2h\" (UniqueName: \"kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h\") pod \"test-curl-7151-139766694350896\" (UID: \"6a595e79-7e28-4b87-aa40-1b33357fcb3a\") " pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:34.928647 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.928626 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsg2h\" (UniqueName: \"kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h\") pod \"test-curl-7151-139766694350896\" (UID: \"6a595e79-7e28-4b87-aa40-1b33357fcb3a\") " pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:34.997202 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:34.997132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:35.122386 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:35.122358 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766694350896"] Jul 14 10:11:35.124402 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:11:35.124369 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a595e79_7e28_4b87_aa40_1b33357fcb3a.slice/crio-0746e1e61442537aed805195c126ec0edc6ce19990f474bca0ac4723e8a65e4c WatchSource:0}: Error finding container 0746e1e61442537aed805195c126ec0edc6ce19990f474bca0ac4723e8a65e4c: Status 404 returned error can't find the container with id 0746e1e61442537aed805195c126ec0edc6ce19990f474bca0ac4723e8a65e4c Jul 14 10:11:35.125718 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:35.125699 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 10:11:35.474114 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:35.474084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766694350896" event={"ID":"6a595e79-7e28-4b87-aa40-1b33357fcb3a","Type":"ContainerStarted","Data":"0746e1e61442537aed805195c126ec0edc6ce19990f474bca0ac4723e8a65e4c"} Jul 14 10:11:36.480060 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:36.480023 2576 generic.go:356] "Generic (PLEG): container finished" podID="6a595e79-7e28-4b87-aa40-1b33357fcb3a" containerID="a297a44c03979e7174f98440ec76b9a6b0c74760d60ef897b5204a9c1154b043" exitCode=0 Jul 14 10:11:36.480509 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:36.480099 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766694350896" event={"ID":"6a595e79-7e28-4b87-aa40-1b33357fcb3a","Type":"ContainerDied","Data":"a297a44c03979e7174f98440ec76b9a6b0c74760d60ef897b5204a9c1154b043"} Jul 14 10:11:37.621799 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.621774 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:37.660825 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.660806 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7151-139766694350896_6a595e79-7e28-4b87-aa40-1b33357fcb3a/test-curl-7151-139766694350896/0.log" Jul 14 10:11:37.683653 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.683622 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7151-139766694350896"] Jul 14 10:11:37.689290 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.689268 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7151-139766694350896"] Jul 14 10:11:37.740231 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.740160 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsg2h\" (UniqueName: \"kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h\") pod \"6a595e79-7e28-4b87-aa40-1b33357fcb3a\" (UID: \"6a595e79-7e28-4b87-aa40-1b33357fcb3a\") " Jul 14 10:11:37.742249 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.742227 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h" (OuterVolumeSpecName: "kube-api-access-dsg2h") pod "6a595e79-7e28-4b87-aa40-1b33357fcb3a" (UID: "6a595e79-7e28-4b87-aa40-1b33357fcb3a"). InnerVolumeSpecName "kube-api-access-dsg2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:11:37.841670 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:37.841640 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsg2h\" (UniqueName: \"kubernetes.io/projected/6a595e79-7e28-4b87-aa40-1b33357fcb3a-kube-api-access-dsg2h\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:38.442372 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:38.442328 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a595e79-7e28-4b87-aa40-1b33357fcb3a" path="/var/lib/kubelet/pods/6a595e79-7e28-4b87-aa40-1b33357fcb3a/volumes" Jul 14 10:11:38.490433 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:38.490407 2576 scope.go:117] "RemoveContainer" containerID="a297a44c03979e7174f98440ec76b9a6b0c74760d60ef897b5204a9c1154b043" Jul 14 10:11:38.490433 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:38.490432 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766694350896" Jul 14 10:11:40.853654 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.853624 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:40.854003 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.853916 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a595e79-7e28-4b87-aa40-1b33357fcb3a" containerName="test-curl-7151-139766694350896" Jul 14 10:11:40.854003 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.853927 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a595e79-7e28-4b87-aa40-1b33357fcb3a" containerName="test-curl-7151-139766694350896" Jul 14 10:11:40.854003 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.853979 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a595e79-7e28-4b87-aa40-1b33357fcb3a" containerName="test-curl-7151-139766694350896" Jul 14 10:11:40.856755 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.856737 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:40.859464 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.859445 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-zhv22\"" Jul 14 10:11:40.863513 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.863479 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:40.963774 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:40.963746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8zwv\" (UniqueName: \"kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv\") pod \"test-curl-7151-139766691504176\" (UID: \"0f70d5e8-709d-4167-954e-a38ff691dcb8\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:41.065057 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:41.065026 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8zwv\" (UniqueName: \"kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv\") pod \"test-curl-7151-139766691504176\" (UID: \"0f70d5e8-709d-4167-954e-a38ff691dcb8\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:41.073306 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:41.073279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8zwv\" (UniqueName: \"kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv\") pod \"test-curl-7151-139766691504176\" (UID: \"0f70d5e8-709d-4167-954e-a38ff691dcb8\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:41.166298 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:41.166247 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:41.289982 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:41.289955 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:41.291992 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:11:41.291959 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f70d5e8_709d_4167_954e_a38ff691dcb8.slice/crio-3d14b9c3c93a0f55c88323cc869b5e0b6fc83ccea389646e10c8075b1b250b80 WatchSource:0}: Error finding container 3d14b9c3c93a0f55c88323cc869b5e0b6fc83ccea389646e10c8075b1b250b80: Status 404 returned error can't find the container with id 3d14b9c3c93a0f55c88323cc869b5e0b6fc83ccea389646e10c8075b1b250b80 Jul 14 10:11:41.503975 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:41.503946 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766691504176" event={"ID":"0f70d5e8-709d-4167-954e-a38ff691dcb8","Type":"ContainerStarted","Data":"3d14b9c3c93a0f55c88323cc869b5e0b6fc83ccea389646e10c8075b1b250b80"} Jul 14 10:11:42.112329 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:42.112238 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:11:42.133256 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:42.116556 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v7dl8_328b120f-2c8a-4462-b843-47ad0e7c9d92/ovn-acl-logging/0.log" Jul 14 10:11:42.509245 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:42.509213 2576 generic.go:356] "Generic (PLEG): container finished" podID="0f70d5e8-709d-4167-954e-a38ff691dcb8" containerID="db06108c3ea9312121267072123dc7732ec2edf336c0aefed8e12b33caf6a6cb" exitCode=0 Jul 14 10:11:42.509389 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:42.509298 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766691504176" event={"ID":"0f70d5e8-709d-4167-954e-a38ff691dcb8","Type":"ContainerDied","Data":"db06108c3ea9312121267072123dc7732ec2edf336c0aefed8e12b33caf6a6cb"} Jul 14 10:11:43.650544 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.650520 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:43.672679 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.672655 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7151-139766691504176_0f70d5e8-709d-4167-954e-a38ff691dcb8/test-curl-7151-139766691504176/0.log" Jul 14 10:11:43.699029 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.698995 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:43.702334 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.702314 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:43.786003 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.785950 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8zwv\" (UniqueName: \"kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv\") pod \"0f70d5e8-709d-4167-954e-a38ff691dcb8\" (UID: \"0f70d5e8-709d-4167-954e-a38ff691dcb8\") " Jul 14 10:11:43.787916 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.787887 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv" (OuterVolumeSpecName: "kube-api-access-p8zwv") pod "0f70d5e8-709d-4167-954e-a38ff691dcb8" (UID: "0f70d5e8-709d-4167-954e-a38ff691dcb8"). InnerVolumeSpecName "kube-api-access-p8zwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:11:43.822839 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.822818 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:43.823103 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.823091 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f70d5e8-709d-4167-954e-a38ff691dcb8" containerName="test-curl-7151-139766691504176" Jul 14 10:11:43.823178 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.823104 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f70d5e8-709d-4167-954e-a38ff691dcb8" containerName="test-curl-7151-139766691504176" Jul 14 10:11:43.823178 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.823174 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f70d5e8-709d-4167-954e-a38ff691dcb8" containerName="test-curl-7151-139766691504176" Jul 14 10:11:43.825671 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.825654 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:43.830164 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.830144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:43.886758 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.886705 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8zwv\" (UniqueName: \"kubernetes.io/projected/0f70d5e8-709d-4167-954e-a38ff691dcb8-kube-api-access-p8zwv\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:43.987930 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:43.987879 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ljp\" (UniqueName: \"kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp\") pod \"test-curl-7151-139766691504176\" (UID: \"c549fb90-2473-4a1a-a641-68f005011bac\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:44.089119 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.089042 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ljp\" (UniqueName: \"kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp\") pod \"test-curl-7151-139766691504176\" (UID: \"c549fb90-2473-4a1a-a641-68f005011bac\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:44.097887 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.097857 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ljp\" (UniqueName: \"kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp\") pod \"test-curl-7151-139766691504176\" (UID: \"c549fb90-2473-4a1a-a641-68f005011bac\") " pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:44.134696 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.134669 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:44.279471 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.279343 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:44.281587 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:11:44.281564 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc549fb90_2473_4a1a_a641_68f005011bac.slice/crio-56de7d31aaccf7e9059898a735a1e77c35dc60f90bb1e404671ec2716561c0b2 WatchSource:0}: Error finding container 56de7d31aaccf7e9059898a735a1e77c35dc60f90bb1e404671ec2716561c0b2: Status 404 returned error can't find the container with id 56de7d31aaccf7e9059898a735a1e77c35dc60f90bb1e404671ec2716561c0b2 Jul 14 10:11:44.441830 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.441803 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f70d5e8-709d-4167-954e-a38ff691dcb8" path="/var/lib/kubelet/pods/0f70d5e8-709d-4167-954e-a38ff691dcb8/volumes" Jul 14 10:11:44.517589 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.517559 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766691504176" event={"ID":"c549fb90-2473-4a1a-a641-68f005011bac","Type":"ContainerStarted","Data":"56de7d31aaccf7e9059898a735a1e77c35dc60f90bb1e404671ec2716561c0b2"} Jul 14 10:11:44.518630 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.518612 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:44.518630 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:44.518596 2576 scope.go:117] "RemoveContainer" containerID="db06108c3ea9312121267072123dc7732ec2edf336c0aefed8e12b33caf6a6cb" Jul 14 10:11:45.525240 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:45.525178 2576 generic.go:356] "Generic (PLEG): container finished" podID="c549fb90-2473-4a1a-a641-68f005011bac" containerID="bd7493f0596e45b59fef762c545e14e798f14a1136783b7baf27b8d193e2158a" exitCode=0 Jul 14 10:11:45.525713 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:45.525275 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7151-139766691504176" event={"ID":"c549fb90-2473-4a1a-a641-68f005011bac","Type":"ContainerDied","Data":"bd7493f0596e45b59fef762c545e14e798f14a1136783b7baf27b8d193e2158a"} Jul 14 10:11:46.661753 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.661731 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:46.683831 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.683810 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7151-139766691504176_c549fb90-2473-4a1a-a641-68f005011bac/test-curl-7151-139766691504176/0.log" Jul 14 10:11:46.708202 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.708170 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:46.709947 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.709929 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8ljp\" (UniqueName: \"kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp\") pod \"c549fb90-2473-4a1a-a641-68f005011bac\" (UID: \"c549fb90-2473-4a1a-a641-68f005011bac\") " Jul 14 10:11:46.711943 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.711918 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7151-139766691504176"] Jul 14 10:11:46.712047 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.712028 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp" (OuterVolumeSpecName: "kube-api-access-s8ljp") pod "c549fb90-2473-4a1a-a641-68f005011bac" (UID: "c549fb90-2473-4a1a-a641-68f005011bac"). InnerVolumeSpecName "kube-api-access-s8ljp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:11:46.810821 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:46.810768 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8ljp\" (UniqueName: \"kubernetes.io/projected/c549fb90-2473-4a1a-a641-68f005011bac-kube-api-access-s8ljp\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:47.536531 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:47.536501 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:47.536714 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:47.536506 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56de7d31aaccf7e9059898a735a1e77c35dc60f90bb1e404671ec2716561c0b2" Jul 14 10:11:47.546602 ip-10-0-129-15 kubenswrapper[2576]: E0714 10:11:47.546570 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7151-139766691504176\" is forbidden: User \"system:node:ip-10-0-129-15.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-15.ec2.internal' and this object" podUID="c549fb90-2473-4a1a-a641-68f005011bac" pod="openshift-ingress/test-curl-7151-139766691504176" Jul 14 10:11:48.441707 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.441672 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c549fb90-2473-4a1a-a641-68f005011bac" path="/var/lib/kubelet/pods/c549fb90-2473-4a1a-a641-68f005011bac/volumes" Jul 14 10:11:48.649723 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.649686 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" podUID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" containerName="main" containerID="cri-o://9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57" gracePeriod=45 Jul 14 10:11:48.900565 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.900543 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:11:48.923955 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.923932 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924065 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924002 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924065 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924028 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924065 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924050 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924233 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924075 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924233 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924134 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngnnm\" (UniqueName: \"kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm\") pod \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\" (UID: \"9de114cc-5e0c-4985-8bb6-36cce96ae58d\") " Jul 14 10:11:48.924429 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924384 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:11:48.924429 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924395 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:11:48.924429 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.924414 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home" (OuterVolumeSpecName: "home") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:11:48.926530 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.926506 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm" (OuterVolumeSpecName: "dshm") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:11:48.926765 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.926739 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm" (OuterVolumeSpecName: "kube-api-access-ngnnm") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "kube-api-access-ngnnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:11:48.926857 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:48.926796 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9de114cc-5e0c-4985-8bb6-36cce96ae58d" (UID: "9de114cc-5e0c-4985-8bb6-36cce96ae58d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025470 2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tmp-dir\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025491 2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-home\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025500 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngnnm\" (UniqueName: \"kubernetes.io/projected/9de114cc-5e0c-4985-8bb6-36cce96ae58d-kube-api-access-ngnnm\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025509 2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-dshm\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025518 2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9de114cc-5e0c-4985-8bb6-36cce96ae58d-tls-certs\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.025524 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.025526 2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9de114cc-5e0c-4985-8bb6-36cce96ae58d-model-cache\") on node \"ip-10-0-129-15.ec2.internal\" DevicePath \"\"" Jul 14 10:11:49.546309 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.546274 2576 generic.go:356] "Generic (PLEG): container finished" podID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" containerID="9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57" exitCode=0 Jul 14 10:11:49.546663 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.546342 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" Jul 14 10:11:49.546663 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.546342 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" event={"ID":"9de114cc-5e0c-4985-8bb6-36cce96ae58d","Type":"ContainerDied","Data":"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57"} Jul 14 10:11:49.546663 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.546452 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p" event={"ID":"9de114cc-5e0c-4985-8bb6-36cce96ae58d","Type":"ContainerDied","Data":"c35416892d05f9836ab65c5dae127423b923bfd33320392a827c687e3a9d7c26"} Jul 14 10:11:49.546663 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.546472 2576 scope.go:117] "RemoveContainer" containerID="9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57" Jul 14 10:11:49.556404 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.556382 2576 scope.go:117] "RemoveContainer" containerID="9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57" Jul 14 10:11:49.556762 ip-10-0-129-15 kubenswrapper[2576]: E0714 10:11:49.556736 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57\": container with ID starting with 9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57 not found: ID does not exist" containerID="9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57" Jul 14 10:11:49.556860 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.556768 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57"} err="failed to get container status \"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57\": rpc error: code = NotFound desc = could not find container \"9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57\": container with ID starting with 9ca1653571a95be6d1d9cc2554bc642bed939680059d001d6c025196934fcf57 not found: ID does not exist" Jul 14 10:11:49.570388 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.570364 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:11:49.573965 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:49.573932 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ad5ead/rate-test-model-ad5ead-kserve-5b78787699-llf8p"] Jul 14 10:11:50.441973 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:11:50.441936 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" path="/var/lib/kubelet/pods/9de114cc-5e0c-4985-8bb6-36cce96ae58d/volumes" Jul 14 10:12:45.625051 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:45.625017 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-695c4bb98d-6fcn5_8a673ea3-65ca-4195-a3ce-d2df44ef0f01/authorino/0.log" Jul 14 10:12:50.626937 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:50.626904 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-794679ff6-6kj66_0dc40375-ac01-420e-a388-ff7fdb6f8ef1/manager/0.log" Jul 14 10:12:50.747770 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:50.747746 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qtlgs_23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75/server/0.log" Jul 14 10:12:51.099881 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:51.099851 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-hskvr_c5e7119a-1eb7-4565-bc0b-93743d28fa2b/copy-manifests/0.log" Jul 14 10:12:51.107230 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:51.107201 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-hskvr_c5e7119a-1eb7-4565-bc0b-93743d28fa2b/manager/0.log" Jul 14 10:12:54.585721 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:54.585693 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-695c4bb98d-6fcn5_8a673ea3-65ca-4195-a3ce-d2df44ef0f01/authorino/0.log" Jul 14 10:12:55.964309 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:55.964281 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-bwlnw_2f2efddb-8715-4afe-afe0-c286ca071483/kube-auth-proxy/0.log" Jul 14 10:12:56.433902 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:56.433872 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7p2m_538ec3f8-7f89-4638-a4e8-935003561218/payload-processing/0.log" Jul 14 10:12:56.990591 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:56.990565 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qzd88_39c8f616-c1ba-4cfa-b3a2-5b440ec865f8/main/0.log" Jul 14 10:12:57.317258 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:57.317175 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-prvsm_4c460381-d56e-400d-a4ae-5e74a73d7884/main/0.log" Jul 14 10:12:57.427357 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:12:57.427333 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-9hjt8_bf437432-a6d6-4a22-aeae-45e8edb9d7c3/main/0.log" Jul 14 10:13:04.751586 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:04.751506 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s2jnc_3ace0aec-617e-4dea-bc85-b2a201718832/global-pull-secret-syncer/0.log" Jul 14 10:13:04.885073 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:04.885045 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pwxws_14481b43-9d55-419e-9072-e2bb2e0f54d8/konnectivity-agent/0.log" Jul 14 10:13:04.906168 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:04.906135 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-15.ec2.internal_44c9368520183df5fbab72347490a409/haproxy/0.log" Jul 14 10:13:09.831745 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:09.831702 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-695c4bb98d-6fcn5_8a673ea3-65ca-4195-a3ce-d2df44ef0f01/authorino/0.log" Jul 14 10:13:12.258704 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:12.258633 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bkx_faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a/node-exporter/0.log" Jul 14 10:13:12.282000 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:12.281976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bkx_faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a/kube-rbac-proxy/0.log" Jul 14 10:13:12.304076 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:12.304055 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-95bkx_faa9d3eb-0a51-4835-ac08-58fcc5d0fe0a/init-textfile/0.log" Jul 14 10:13:14.979532 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979502 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6"] Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979792 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c549fb90-2473-4a1a-a641-68f005011bac" containerName="test-curl-7151-139766691504176" Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979804 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c549fb90-2473-4a1a-a641-68f005011bac" containerName="test-curl-7151-139766691504176" Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979822 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" containerName="main" Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979828 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" containerName="main" Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979875 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de114cc-5e0c-4985-8bb6-36cce96ae58d" containerName="main" Jul 14 10:13:14.979900 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.979884 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c549fb90-2473-4a1a-a641-68f005011bac" containerName="test-curl-7151-139766691504176" Jul 14 10:13:14.982725 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.982705 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:14.985488 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.985463 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s29td\"/\"openshift-service-ca.crt\"" Jul 14 10:13:14.985598 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.985489 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s29td\"/\"kube-root-ca.crt\"" Jul 14 10:13:14.986347 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.986324 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s29td\"/\"default-dockercfg-zq65n\"" Jul 14 10:13:14.992019 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:14.991995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6"] Jul 14 10:13:15.149885 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.149857 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-podres\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.150024 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.149901 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-sys\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.150024 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.149921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzvfl\" (UniqueName: \"kubernetes.io/projected/7565aef7-c32a-4482-835d-57d2f10d444a-kube-api-access-wzvfl\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.150024 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.149946 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-lib-modules\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.150024 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.150003 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-proc\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251461 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-proc\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251461 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-podres\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-sys\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251527 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzvfl\" (UniqueName: \"kubernetes.io/projected/7565aef7-c32a-4482-835d-57d2f10d444a-kube-api-access-wzvfl\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251550 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-podres\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251554 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-lib-modules\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251556 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-proc\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251587 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-sys\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.251678 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.251647 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7565aef7-c32a-4482-835d-57d2f10d444a-lib-modules\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.260866 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.260837 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzvfl\" (UniqueName: \"kubernetes.io/projected/7565aef7-c32a-4482-835d-57d2f10d444a-kube-api-access-wzvfl\") pod \"perf-node-gather-daemonset-h2cz6\" (UID: \"7565aef7-c32a-4482-835d-57d2f10d444a\") " pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.293368 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.293342 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.421920 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.421897 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6"] Jul 14 10:13:15.424076 ip-10-0-129-15 kubenswrapper[2576]: W0714 10:13:15.424050 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7565aef7_c32a_4482_835d_57d2f10d444a.slice/crio-145de0a83a17898402ea87701d0c74a44d17a781917f1cab775607f32d8d9ad1 WatchSource:0}: Error finding container 145de0a83a17898402ea87701d0c74a44d17a781917f1cab775607f32d8d9ad1: Status 404 returned error can't find the container with id 145de0a83a17898402ea87701d0c74a44d17a781917f1cab775607f32d8d9ad1 Jul 14 10:13:15.894274 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.894169 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" event={"ID":"7565aef7-c32a-4482-835d-57d2f10d444a","Type":"ContainerStarted","Data":"6b6c165b0ee0d8e32c5f9e04855871159868f455e104f5051d1b6cced4080402"} Jul 14 10:13:15.894274 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.894239 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" event={"ID":"7565aef7-c32a-4482-835d-57d2f10d444a","Type":"ContainerStarted","Data":"145de0a83a17898402ea87701d0c74a44d17a781917f1cab775607f32d8d9ad1"} Jul 14 10:13:15.894274 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.894264 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:15.911018 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:15.910975 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" podStartSLOduration=1.910960662 podStartE2EDuration="1.910960662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:13:15.910447508 +0000 UTC m=+2794.388482935" watchObservedRunningTime="2026-07-14 10:13:15.910960662 +0000 UTC m=+2794.388996086" Jul 14 10:13:17.426123 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:17.426096 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w558q_ad29e033-9aa7-4d33-8cb0-1eef8d354844/dns/0.log" Jul 14 10:13:17.447625 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:17.447601 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-w558q_ad29e033-9aa7-4d33-8cb0-1eef8d354844/kube-rbac-proxy/0.log" Jul 14 10:13:17.496385 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:17.496360 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw8nq_dead7e1a-456c-4dc9-aba6-799c6b8c945f/dns-node-resolver/0.log" Jul 14 10:13:18.073089 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:18.073051 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c647c656d-c22xh_98ecac76-e91d-4f50-893c-7c4f891530ba/registry/0.log" Jul 14 10:13:18.124245 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:18.124211 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tkgk_6ed79517-5b5e-4b46-ab41-a0f2831fe646/node-ca/0.log" Jul 14 10:13:19.265922 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:19.265883 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-bwlnw_2f2efddb-8715-4afe-afe0-c286ca071483/kube-auth-proxy/0.log" Jul 14 10:13:19.456265 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:19.456231 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7p2m_538ec3f8-7f89-4638-a4e8-935003561218/payload-processing/0.log" Jul 14 10:13:20.198038 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:20.197996 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8vwt_d39ef447-4a78-4e92-92a2-4f0252a4db58/serve-healthcheck-canary/0.log" Jul 14 10:13:20.774256 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:20.774231 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtqgg_ac2bb1f4-1f94-44e9-ae16-79ade319531a/kube-rbac-proxy/0.log" Jul 14 10:13:20.796381 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:20.796357 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtqgg_ac2bb1f4-1f94-44e9-ae16-79ade319531a/exporter/0.log" Jul 14 10:13:20.818043 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:20.818015 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtqgg_ac2bb1f4-1f94-44e9-ae16-79ade319531a/extractor/0.log" Jul 14 10:13:21.907928 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:21.907899 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s29td/perf-node-gather-daemonset-h2cz6" Jul 14 10:13:23.201891 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:23.201855 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-794679ff6-6kj66_0dc40375-ac01-420e-a388-ff7fdb6f8ef1/manager/0.log" Jul 14 10:13:23.224533 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:23.224504 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qtlgs_23f88bb4-f8ff-4ea3-b909-6c3c4fec4c75/server/0.log" Jul 14 10:13:23.373625 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:23.373596 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-hskvr_c5e7119a-1eb7-4565-bc0b-93743d28fa2b/manager/0.log" Jul 14 10:13:23.396971 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:23.396946 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-644f4b6bc9-hskvr_c5e7119a-1eb7-4565-bc0b-93743d28fa2b/copy-manifests/0.log" Jul 14 10:13:24.917348 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:24.917280 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56954f6f7d-498qk_4a166add-ba0d-433b-a1be-1d0a8f5f7766/manager/0.log" Jul 14 10:13:25.601323 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:25.601296 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-bwlnw_2f2efddb-8715-4afe-afe0-c286ca071483/kube-auth-proxy/0.log" Jul 14 10:13:25.769562 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:25.769524 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7p2m_538ec3f8-7f89-4638-a4e8-935003561218/payload-processing/0.log" Jul 14 10:13:27.203969 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:27.203946 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8569779fcf-bwlnw_2f2efddb-8715-4afe-afe0-c286ca071483/kube-auth-proxy/0.log" Jul 14 10:13:27.361371 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:27.361343 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-d7p2m_538ec3f8-7f89-4638-a4e8-935003561218/payload-processing/0.log" Jul 14 10:13:30.141033 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:30.141002 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vjl28_de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69/migrator/0.log" Jul 14 10:13:30.161020 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:30.160996 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-vjl28_de2bbe9c-4c6f-41e9-a6af-7398bd3e3b69/graceful-termination/0.log" Jul 14 10:13:31.552646 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:31.552620 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-76qwn_bcd140d7-76f7-4094-a426-093973d47036/kube-multus/0.log" Jul 14 10:13:32.027916 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.027891 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/kube-multus-additional-cni-plugins/0.log" Jul 14 10:13:32.058993 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.058976 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/egress-router-binary-copy/0.log" Jul 14 10:13:32.088473 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.088451 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/cni-plugins/0.log" Jul 14 10:13:32.114391 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.114374 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/bond-cni-plugin/0.log" Jul 14 10:13:32.136439 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.136419 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/routeoverride-cni/0.log" Jul 14 10:13:32.159824 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.159802 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/whereabouts-cni-bincopy/0.log" Jul 14 10:13:32.181958 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.181940 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gk9tm_088c7d44-3d7c-4573-8850-8af99b26bb54/whereabouts-cni/0.log" Jul 14 10:13:32.310694 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.310644 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qh6lf_bc0962d2-8ea1-4296-8e39-a4d59d12d652/network-metrics-daemon/0.log" Jul 14 10:13:32.335387 ip-10-0-129-15 kubenswrapper[2576]: I0714 10:13:32.335368 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-qh6lf_bc0962d2-8ea1-4296-8e39-a4d59d12d652/kube-rbac-proxy/0.log"